Managin Service Quality with the Balanced Scorecard
In:
Submitted By sportesse16 Words 4676 Pages 19
MANAGING SERVICE QUALITY WITH THE BALANCED SCORECARD
Roswitha Poll
THE DATA IN THE CONTROLLING SYSTEM
Traditionally, libraries have collected statistical data about their collections, acquisitions, lending, and inter-lending activities. In time, the number of statistics was enlarged and differentiated, and in many cases, it now comprises several hundred data points. These range from the number of incunabula or microforms in the collection, the expenditure on preservation or buildings to the number of issues made, claims and reservations placed or visits made to exhibitions and special events. These statistics are, for the most part, collected nationally, but libraries also tend to collect additional statistics, (e.g. for special tasks and activities like legal deposit right, special collections, or services for special user groups). All those statistical data could be used as steering instruments for library management. But such use is more often accidental than systematic, and many data are collected laboriously without ever being evaluated or used. For several decades libraries have tried to assess, not only the quantity of their resources and activities, but also the quality – the “goodness” of a library’s services and products. Performance measures for libraries have been developed and tested in national and international projects and standardized in an international standard (ISO 11620, 1998). Though there are lists of recognized and established performance indicators, such indicators have not been collected and used systematically, like in the national collection of statistics, but rather in the evaluation process of a single library.
Statistics, as well as performance indicators, were developed for use in the traditional library that has print collections, reading rooms and lending services. The growing importance of electronic services in libraries has led to a revision of both statistics and performance indicators. The international standard of library statistics has been revised and enlarged to include the data of the “digital library” (ISO 2789, 2003), and a working group of ISO (the International Organization of Standardization) has edited a Technical Report about performance indicators for electronic library services. (ISO TR 20983, 2003) Another sector of management data has evolved over the last few years: cost data. Libraries have always registered data relating to their income and expenditure. But the general demand for transparency of costs has led to questions like: What are the costs of each single service or product of a library (e.g. one issue, one reference question answered)? How do the costs of a service or product split up as to staff costs, administrative costs, equipment etc.? More and more libraries are involved in cost analysis projects of programs within their institutions, or are trying to analyze their costs in order to present reliable data when applying for funds or allocating resources. Models for cost analysis in libraries have been developed and tested (Ceynowa & Coners, 2003; Poll, 2000) and will probably be used widely in the future. Nowadays, there is an immense pool of management data available in libraries, to include statistics on resources, services, use, and cost data and combined data like performance indicators for the quality of library services. The quantity, diversity and complexity of the data stress the need for an integrated system designed to make this management data useful for evaluating programs, developing strategy, and initiating action.
THE BALANCED SCORECARD MODEL
A German project, sponsored by the German Research Council (DFG), has developed an integrated quality management system for academic libraries. The project was chaired by the University and Regional Library of M¨ nster and included u as partners the Bavarian State Library in Munich and the State and University Library in Bremen. The three libraries are among the largest in Germany, each with special tasks, activities, and operating conditions. Thus, the project could rely on a broad and differentiated view of management issues in academic libraries. The project started in June 1999 and finished in the autumn of 2001. The results have been published in a handbook, with a software package designed to facilitate
Managing Service Quality with the Balanced Scorecard
215
Fig. 1. Managing with the Balanced Scorecard.
the collection of data and the management process (Ceynowa & Coners, 2002). The project partners decided to use the Balanced Scorecard (Kaplan & Norton, 1992, 1996) as a tool in developing this management system. This concept was originally developed for the commercial sector. It “translates” the planning perspective of an institution (mission, strategic vision and goals) into a system of performance indicators that covers all important perspectives of performance: finances, users, internal processes and improvement activities (Fig. 1). The system thus integrates: financial and non-financial data; input and output data; the external perspective (funding institutions, users) and the internal perspective (processes, staff);
216
ROSWITHA POLL
goals and measures taken; causes and results. The basic model of the Balanced Scorecard, adapted to the conditions of academic libraries, deviates from the original model in placing the user perspective rather than the financial perspective foremost, because libraries do not strive for maximum financial gain, but for best service. The following sections show the different performance perspectives of the Balanced Scorecard Model, the respective indicators for each perspective, and the actual values found in the project libraries. Performance indicators have not yet been used systematically in German academic libraries, and a broad comparison of output measures is not feasible at this time. Therefore, the project libraries decided to publish their own data, collected during the course of the project (the year 2000), in order to help libraries using the Balanced Scorecard Model estimate the position of their own data collection efforts. The three project libraries vary greatly as to mission, structure and clientele: The State and University Library (SuUB) Bremen has a one-tier library system on a campus. The University and Regional Library (ULB) M¨ nster has a two-tier library u system with 159 institute libraries. The Bavarian State Library (BSB) M¨ nchen does not serve a university, but the u general academic community in a broad frame. As a result, a large number of the indicators are not applicable for comparison between the three libraries. The indicator “market penetration,” for instance, shows lower values for the two-tiered system in M¨ nster than for Bremen, since u the central library shares this task in M¨ nster with the institute libraries. It does u not apply at all to the Bavarian State Library because that library does not serve a clearly defined population. Libraries testing the performance indicators of the model for their own use will have to decide which of the three library types considered is best comparable with their own mission, structure and clientele.
The User Perspective The indicators chosen for the user perspective correspond to the fundamental goals of reaching as large a part of the population as possible and of satisfying their informational needs with the services offered. Market penetration = Percent of the population registered as actual users. For an academic library, the population to be served will probably consist of the members of the institution (university, scientific institute etc.) it is set up
Managing Service Quality with the Balanced Scorecard
217
to serve. Actual users are those that have transacted at least one loan during a specified time period (usually one year). It would, of course, be even more interesting to assess the proportion of the population who made use of other library services during a specified time, including electronic services and remote use. But, for many libraries, this is still difficult to measure. Market penetration for the test libraries was: SuUB Bremen ULB M¨ nster u BSB M¨ nchen u 74.5% 71.2% not applicable
The project libraries chose to assess “actual users” instead of “registered users” for this indicator, since in many libraries students’ enrollment automatically includes registration for library services and market penetration could not be measured. User satisfaction rate This indicator is assessed using satisfaction surveys on a 5-point-scale. Overall satisfaction with the library is generally assessed in a broader survey that asks for satisfaction with different library services and for frequency of use of those services. The data showed that the average user satisfaction rating was: SuUB Bremen ULB M¨ nster u 2.39 2.40
The BSB M¨ nchen restricted its satisfaction survey to the users of the reading u rooms and did not produce comparable data. User satisfaction with opening times Open times of libraries have always been regarded as an important quality indicator. But the library’s location and surroundings, mission and population may influence the need for a certain set of operating hours. Therefore user satisfaction with the opening hours of the library was chosen as the more relevant indicator. Satisfaction ratings for the hours of operation of the libraries were: SuUB Bremen ULB M¨ nster u 1.59 2.15
The BSB M¨ nchen restricted the survey to the users of its reading rooms. u Library visits per member of the population The visits to the library premises (counted by turnstile, electronic counter etc.) per year are compared to the number of members of the population. “Virtual visits” (cases of remote use) by the population should also be counted and evaluated separately, as activities and length of time differ too much from those of traditional visits. As yet, however, most libraries are not able to differentiate
218
ROSWITHA POLL
exactly between remote use coming from members of the population and other use, making comparison difficult. The number of visits per year per member of the population was: SuUB Bremen ULB M¨ nster u BSB M¨ nchen u 13.2 visits 22.0 visits not applicable
Immediate availability This indicator is measured based on the percent of immediate loans among the total number of loans made (including reservations and ILL) The indicator is where A = number of loans minus those made by reservation B = number of all loans from the library collections × (including those by reservation) + ILL − loans The indicator shows whether the collection covers all topics asked for by users and whether there are sufficient copies of specific titles required. By this measure, immediate availability rates were: SuUB Bremen ULB M¨ nster u BSB M¨ nchen u 92.6% 84.8% 95.0% (data from 1999) A ×100%, B
The results clearly showed a high number of books on loan when requested in the M¨ nster library, a library serving a university that has one of the largest u populations among German universities (45,000 students). Two indicators assess the use of electronic services offered by the library and the growing portion of that use that is coming from outside the library building: Percent of the population using the electronic library services This indicator is assessed using a survey. Electronic services include the OPAC, the library’s website, electronic databases, journals, and other electronic documents, and electronic document delivery. For the libraries studied, the percentage of patrons using these services were:
Managing Service Quality with the Balanced Scorecard
219
SuUB Bremen ULB M¨ nster u BSB M¨ nchen u
94.2% 94.5% between 46.3–64.0%
Bremen assessed the data in a general satisfaction survey that queried only users coming into the library. M¨ nster assessed the data in a special telephone survey, u choosing a relevant sample of its population. M¨ nchen assessed the data in its u survey of reading-room users. All libraries included more detailed questions as to the use of certain services (e.g. electronic journals). Percent of remote accesses to electronic library services of all accesses This indicator assesses what percentage of electronic services use was from outside the library. One access (or session) is defined as one established connection to an electronic library service. Data collection in the project was restricted to the OPAC and the website and was developed by evaluating the log files of the web server. Remote use of the OPAC and website was as follows: SuUB Bremen ULB M¨ nster u BSB M¨ nchen u OPAC Website OPAC Website OPAC Website 49.9% 52.3% 62.3% 40.1% 40.4% 46.7%
The data of M¨ nchen refer to the year 1999, the other data to the year 2000. u
The Financial Perspective The indicators for the financial perspective answer the question as to whether the library is operating in a cost-effective way. The goals comprise low costs per instance of use or per product and a high proportion of the total budget spent on the print and electronic collection. Total costs of the library per active user The total costs comprise staff costs, collection building and maintenance, administrative costs, operating costs, depreciation and rent. If the library has no cost accounting procedure, the total expenditure could be used as a substitute. “Active users” were chosen for this indicator instead of “members of the population” in order to show the library’s options for optimizing cost data. If market penetration rises and there are more active users, costs per user will automatically go down. Costs calculated using this measure were:
220
ROSWITHA POLL
SuUB Bremen ULB M¨ nster u BSB M¨ nchen u
322.83 d 313.14 d 350.37 d
M¨ nchen used the expenditure data of 1999 instead of cost data. Bremen u and M¨ nster apply cost accounting procedures and used data for the year u 2000. Total costs of the library per library visit The costs of individual library visits were: SuUB Bremen ULB M¨ nster u 27.03 d 11.22 d
For M¨ nster, only the visits in the central library were counted and compared u with the central library’s costs. In M¨ nchen, only the visits to the reading-rooms u were counted. Thus, the data could not be compared to the overall costs there. Acquisitions expenditure compared to the total costs of the library Acquisitions expenditure comprises expenditure for print, electronic, and other media (including licenses and cost-per-view, if paid by the library), but excludes expenditures for preservation. The percentages of the entire library budget at individual institutions were: SuUB Bremen ULB M¨ nster u BSB M¨ nchen u 40% 20% 33.4% (data from 1999)
Bremen’s high percentage of acquisition expenditure shows the possibilities provided within the library’s global budget to shift resources between staff and other budget items. Percent of staff costs per library service/product to total staff costs The model differentiates as to the following services/products: collection building (including cataloguing); information services; user services 1: loans/reading-rooms; user services 2: document delivery/ILL; preservation; administration/IT-services; special services/projects. Staff costs were counted in standardized average rates. Staff members were classified as to their assignment to departments/working groups, not as to
Managing Service Quality with the Balanced Scorecard
221
specified activities. As a result, short time work in other departments is not represented. The percentage of staff time in each library assigned to each activity was: • collection building • information services • user services 1 • user services 2 • preservation • administration/IT-services • special services/projects SuUB Bremen ULB M¨ nster u BSB M¨ nchen u SuUB Bremen ULB M¨ nster u BSB M¨ nchen u SuUB Bremen ULB M¨ nster u BSB M¨ nchen u SuUB Bremen ULB M¨ nster u BSB M¨ nchen u SuUB Bremen ULB M¨ nster u BSB M¨ nchen u SuUB Bremen ULB M¨ nster u BSB M¨ nchen u SuUB Bremen ULB M¨ nster u BSB M¨ nchen u 30.3% 27.2% 44.8% 27.6% 8.3% (all user services): 43.9% 13.0% 16.6% (all user services): 43.9% 7.4% 10.2% (all user services): 43.9% 1.3% 2.2% 4.9% 13.6% 15.8% 6.4% 6.8% 19.7% not assessed
The results show, for instance, the high percentage of time devoted to collection building in the BSB M¨ nchen with its large special collections, and a high u percentage for special services/projects in M¨ nster with special collection tasks u and cost-intensive services for the institute libraries in the system. A last indicator shows the allocation of resources to the electronic library: Percent of acquisitions expenditure spent on electronic media Acquisitions expenditure for electronic media includes licenses, the library’s own expenditure in consortia, and pay-per-view costs paid by the library. These constitute the following percentages of the acquisitions budget: SuUB Bremen ULB M¨ nster u BSB M¨ nchen u 7.2% 3.8% 3.3% (data from the year 1999)
222
ROSWITHA POLL
The Perspective of Processes For the perspective of processes, the underlying goals are to organize all processes in such a way that, despite budget restrictions, space for investment into new developments and improvement of service is allowed. The indicators pick out background activities as examples of process organization. Acquired media per staff year The processing department comprises all activities from acquisition and cataloguing to binding and shelving the new media. Book processing is seen here as a typical example for the effectiveness of background processes. Staff members in the processing department are counted in FTE (full time equivalent). Electronic media are not included in this indicator. The number of items processed per person were: SuUB Bremen ULB M¨ nster u BSB M¨ nchen u 3,748 media 3,057 media 2,797 media (data from the year 1999)
While Bremen und M¨ nster have integrated acquisition and cataloguing in a u processing department, M¨ nchen still had two separate departments in 1999. u Average media processing time Processing time is seen as the time between the day a document arrives in the library and the day it becomes accessible for users. In the project, measuring was restricted to purchased print monographs, and the average processing times were: SuUB Bremen ULB M¨ nster u BSB M¨ nchen u 25.1 days 17.5 days 40.1 days
The libraries measured processing time as to the separate stages of processing (e.g. inventory, cataloguing, subject cataloguing, technical treatment, shelving). Books given to commercial firms for binding were excluded. In all libraries, subject cataloguing proved to be the most time-consuming part of the process. Number of stages involved in providing a product/service (for every library service/product) This indicator tries to determine how well processes are organized and streamlined. Processing print monographs and document delivery services were evaluated as typical examples. One person performing several activities in a sequence for the same product was seen as one stage, but a different person
Managing Service Quality with the Balanced Scorecard
223
taking over or the same person taking over again after somebody else was seen as different stages. The number of stages involved in various processes were: Processing print monographs (purchased monographs only) SuUB Bremen 15 ULB M¨ nster u 13 BSB M¨ nchen u 19 Electronic document delivery SuUB Bremen 7 ULB M¨ nster u 4–6 BSB M¨ nchen u 5–7 Again, one indicator was chosen to show the allocation of resources to the electronic services: Percent of all staff costs spent on electronic services Library staff involved in planning, maintaining, providing, and developing electronic services is calculated in FTE (full time equivalent). The definition does not include staff in user services (e.g. reference and training services, staff cataloguing electronic media, and the staff involved in developing the contents of the library website). Only regular staff was counted, not persons working on projects or student assistants. The percentage of staff costs devoted to this area were: SuUB Bremen ULB M¨ nster u BSB M¨ nchen u 3.9% 7.5% 6.7% (data from 1999)
The Perspective of Potentials The last perspective, named “potentials,” describes the capability of the library to cope with future challenges and addresses its ability to change and improve. The institution’s engagement for the library is indicated by the budget it allocates to the library; staff as the main factor for all development, is represented by two indicators for learning and engagement. Library budget as percent of the institution’s budget Project funding is excluded. The indicator measures the budget as a percentage of the university budget, not the specific amount.
224
ROSWITHA POLL
The percentage for these institutions is: SuUB Bremen ULB M¨ nster u BSB M¨ nchen u 5.9% 4.5% not applicable
Percent of the library’s expenditure from project funding (special grants) or income generation “Special grants” comprise all funding the library gets by special claims, outside of the “normal” budget; funding for staff is included. This indicator assesses whether the library is successful in claiming additional funding or in generating income and is expressed as a percentage of library expenditures. For these institutions, the percentages are: SuUB Bremen ULB M¨ nster u BSB M¨ nchen u 4.5% 7.4% 15.9% (data from 1999)
Number of training lessons per staff member This indicator assesses the average number of times each staff member has participated in training lessons. In the project, the data were also assessed per service area (collection building, user services, . . .) SuUB Bremen ULB M¨ nster u BSB M¨ nchen u 0.36 lessons 1.08 lessons 0.69 lessons (data from 1999)
Only external training (not in-house) was considered for the indicator. For inhouse training there is often no exact boundary between formal and informal training lessons, so it was excluded. Number of short-time illnesses per staff member A short-time illness is an absence from one to three days. In German public service, such short-time absence need not be accounted for by a medical certificate. Short-time absence is here seen as an indicator of motivation. Data collection in this case must be discussed with staff representatives and must be strictly anonymous. As one of the project libraries did not agree with this indicator and another could not get permission for publishing the data, no data are mentioned here. An alternative indicator for staff motivation might be the fluctuation rate. The best indicator – staff satisfaction rate, assessed by a survey – was regarded as too time-consuming for the project.
Managing Service Quality with the Balanced Scorecard
225
STRATEGY WITH THE BALANCED SCORECARD
One great advantage of the Balanced Scorecard is its ability to visualize relationships of cause and effect between target values, evaluation data and actions taken. The following example shows the planning process from the definition of goals and target values and the choice of adequate indicators to the actions that the library takes to achieve the target value (Fig. 2). As the mission of academic libraries is identical in many aspects, the indicator system within the project might be used as a reference model for benchmarking purposes. Individual differences between libraries can be expressed by different target values and operational actions. Thus, a library whose main task is to provide basic information for students will establish the use of electronic media by offering multimedia-learning material. A special research library, on the other hand, would perhaps offer its scientific journals in electronic form to achieve the same result. Despite such differences, benchmarking is possible. The implementation and continuous use of the Balanced Scorecard demands a large set of data. In the project, a special tool called Library Audit was developed that is based on a system of data analysis (OLAP) and that allows for the multidimensional and flexible analysis of data collections. The library in M¨ nster has already filed the Library Audit with u several thousand bits of data, classified as to the library’s products and services. Benchmarking data of other libraries are added continuously. Many of these data will not be used in the strategic evaluation of the Balanced Scorecard, but the large data pool can be useful in addressing many operational problems.
Fig. 2. The Balanced Scorecard.
226
ROSWITHA POLL
The number of indicators for the Balanced Scorecard has been purposely kept small in order to avoid a flood of data without direct relevance for strategic management. When choosing the indicators for the Balanced Scorecard, the project libraries oriented themselves using the concept of the hybrid library that combines electronic and traditional library services in a comprehensive function. Structuring and implementing a scorecard model for a library demands a clear formulation of mission and strategic goals – a duty that has not yet been performed by every academic library. The most important issue in the integrated controlling concept is not to look at different quality aspects separately, but to keep them all in view. An example shows the steps of measuring quality in collection building: (1) The costs per document processed are low. Does that mean that there are backlogs? (2) Processing time proves quick and adequate. Processes are well organized. But perhaps there is no time for claiming overdue orders? (3) Claiming is done regularly and in good time. Maybe staff is overworked, and absence rates due to illness are rising? (4) Illness rates are quite normal, and a staff satisfaction survey shows high satisfaction with the job. (5) Everything looks fine. But collection use is going down, and a user survey shows dissatisfaction with the collection: Apparently, much well organized labor has been spent on the wrong material. Service quality has many aspects – the Balanced Scorecard integrates them, providing a tool for the assessment of programs.
CONCLUSION
Libraries as yet are comparatively free to choose their own models of evaluation, though evaluation and cost analysis have been made compulsory in many institutions of higher education. The scorecard architecture developed in our project is an attempt to connect vision and mission statement, strategy and goals, programs for action, and indicators for results in a consistent management model. The Balanced Scorecard focuses exclusively on indicators with strategic relevance. Thus the model compels library managers to restrict themselves to measures of the critical issues of service quality. The choice of certain indicators and the priority for certain aspects will probably be questioned by some libraries, and the workload connected with assessing the data might detain others from using the model. Libraries could, of course, exchange some indicators for others, though effort would be required to insure that relevant data for controlling the chosen variables are available.
Managing Service Quality with the Balanced Scorecard
227
The scorecard developed in this project has already influenced management issues in German academic libraries. The university libraries in North RhineWestphalia have agreed on collecting ten “core data” sets that rely very much on the scorecard indicators. The scorecard will probably also be taken into consideration when building up a benchmarking-system for German academic libraries. The Bertelsmann Foundation has for several years been sponsoring a benchmarking project for public libraries with 18 indicators (BertelsmannStiftung). The foundation is now starting a new project for academic libraries together with the German Library Association (Deutscher Bibliotheksverband). The goal is to define a concise set of indicators for controlling and benchmarking and to initiate evaluation and benchmarking procedures on a national scale.
REFERENCES
¨ Bertelsmann-Stiftung, Bereich Offentliche Bibliotheken: Bibliotheksindex (http://www.bix-bibliotheksindex.de). Ceynowa, K., & Coners, A. (2003). Cost management for university libraries. Frankfurt a. M.: Klostermann. Ceynowa, K., & Coners, A. (2002). Balanced scorecard fur wissenschaftliche Bibliotheken (Balanced Scorecard for Academic Libraries) Frankfurt a. M.: Klostermann (Zeitschrift fur Bibliothekswesen und Bibliographie. Sonderheft 82). Kaplan, R. S., & Norton, D. P. (1992). The balanced scorecard – measures that drive performance. Harvard Business Review, 70, 71–79. Kaplan, R. S., & Norton, D. P. (1996). The balanced scorecard: Translating strategy into action. Boston: Harvard Business School. ISO 11620 (1998). Information and documentation – performance indicators for libraries. ISO 2789 (2003). Information and documentation – international library statistics. ISO TR 20983 (2003). Information and documentation – performance indicators for electronic library services – technical report. Poll, R. (2000). The costs of quality: Cost analysis and cost management as counterpart to performance measurement. In: Proceedings of the 3rd Northumbria international conference on performance measusrement in libraries and information services (pp. 43–52). Newcastle upon Tyne: Information North.
