1. 1 Performance Management Presentation Provide Library Services Team Members: Ted Crump, Rosalie Stroman, Jennifer Vrettos, and Susan Whitmore (Team Leader) ORS National Institutes of Health January 21, 2004

2. 2 Table of Contents Main Presentation PM Template ………………….…………………………..……….5 Customer Perspective……………………….…………..….…….7 Internal Business Process Perspective……………………………………………..………16 Learning and Growth Perspective…………………………………………….….……22 Financial Perspective…………………………………..…………25 Conclusions and Recommendations……………………………………………..31

3. 3 Table of Contents (cont.) Appendix Customer Perspective C1a: Percent of current print journals in electronic format……………..…38 C1b: Percentage of license agreements that cover all NIH……………....39 C1c: Percentage of document requests filled in-house…..……………….40 C1d: Number of items translated by government staff and contractors…41 C2a: Number of web-based instruction modules………………………..…43 C2b: Training attendance by type……………………………………………44 C2c: Training evaluation scores…………………………………………..…45 C3a: Customer Scorecard ratings for translation service and mediated database search service..………………………………….…..48 C3c: Digital reference service evaluation ratings…………………………..54 C4: Customer segmentation data……………………………………..……..56 Internal Business Process Perspective IB1: Number and type of special projects contracted for…………..…..…63 IB2: Number of new IT systems implemented…………………………..…65 IB3a: Percent of document requests filled in-house in 3 business days……………………………………………………………………….….69 IB3b: Percentage of documents translated by library staff within 3 business days…………………….…….……………………..…..70 IB3c: Percent of tutorial requests responded to within 1 business day………………………………………………….….….71

4. 4 Table of Contents (cont.) Appendix Learning and Growth Perspective LG1a: Percentage of staff members with IDPs completed by February 28 each fiscal year……………………………………...........…74 LG1b: Percentage of staff completing 80% of activities listed in IDP.......................................................................................................75 LG1c: Percentage of training modules completed on schedule…………76 Financial Perspective F1: Change in unit cost of information…………….……………………….79 F2a: Change in unit cost of custom services…………………………..….82 F2b: No. of contracts/IAGs……………………………………………….….82 F3a: Change in unit cost of providing copies of publications…………….82 F3b: No. of document requests filled……………………………………….82 F4a: Change in unit cost of translating services…………………….…….83 F4b: No. of documents translated…………………………………….…….83 F5a: Change in unit cost of providing primary library services……….….83 F5b: No. of units of information provided…………………………….…….83 F6a: Change in unit cost of providing self-service copiers…………...….83 F6b: No. of visits to self-service copy center………………………..…….83

5. 5

6. 6 Relationship Among Performance Objectives

7. 7 Customer Perspective

8. 8

9. 9 C1a: Percent of current print journals in electronic format Target: Provide 100% of current journals in electronic format Eighty-one percent of the journals are now available in electronic format (as compared to 75% in FY02 and 52% in FY01)

10. 10 Customer Perspective (cont.)

11. 11 C3a: Customer Scorecard ratings for mediated database searching service Mediated database searching service was evaluated from mid-October to mid-December of 2003 using paper or electronic survey forms Forty surveys were distributed and 14 were returned for a return rate of 35% The surveys asked users of the service to rate customer service and search results on a 10 point scale with 1 being unsatisfactory and 10 being outstanding Customers were very satisfied with the service

12. 12 C3a: Customer Scorecard ratings for mediated database searching service – Customer Service

13. 13 C3a: Customer Scorecard ratings for mediated database searching service – Search Results

14. 14 Customer Perspective (cont.)

15. 15 C4: Customer segmentation data Customer segmentation data is presented in the Appendix for four services: Provide copies of publications Provide self-service copiers Translate documents Provide custom research assistance As in past years, customers are spread across all IC’s Heavier use of services by larger IC’s and groups most directly engaged in patient care and biomedical research The Library serves not only NIH but also FDA/CBER on the Bethesda campus and provides document delivery to several groups including NIST and CDC.

16. 16 Internal Business Perspective

17. 17 Internal Business Perspective

18. 18 IB2: Number of new IT systems implemented There were four areas of concentration in FY03: HHS Library Consortium SFX or Full-Text Plus! Relais Content Management Software The major areas of concentration for FY04 are: Continue with the development of the HHS Consortium, Full- Text Plus!, Relais, and CMS Implement MetaLib Enable wireless access in the Library

19. 19 New IT systems implemented in FY03 HHS Library Consortium –In FY03 the Consortium jointly purchased the Nature journals, Science, and Science Express at a considerable savings to NIH Full-Text Plus! – This software, which links various services and resources available from the Library, was implemented for the online journals, the Web of Science and Ovid databases, the new online document request form, and other resources in FY03 Relais – In FY03, phase II of the Relais system was implemented with the integration of the web-based request form and the Web of Science requests. Through Relais the processing of document requests has been considerably streamlined. A process that used to have eight steps now has only three with a turn around time of 1- 3 business days Content Management Software has been implemented for the Library’s Intranet and public websites resulting in a diversification of responsibility for the sites. The redesigned public website will be unveiled on February 10, 2004

20. 20 Internal Business Perspective (cont.)

21. 21 IB3b: Percentage of documents translated by library staff within 3 business days Target: Translate document requests in 3 business days Results: In FY03 an average 55% of documents were translated in 3 business days. In FY04 the translation speed was measured for Library translators and contractors separately. In FY04 Library translators translated 55% of documents in 3 days and contractors translated 26% of documents in 3 days. Efforts to improve the turn around time for translating documents are complicated by: Translation requests are increasing and the current staff are already working as efficiently as possible The more translations that are contracted out, the more time the Library translation staff must spend checking contracted translations thereby reducing the time they have to translate

22. 22 Learning and Growth Perspective

23. 23 Learning and Growth Perspective

24. 24 LG1a: Percentage of staff members with IDPs completed by February 28 each fiscal year Staff are asked to develop Individual Development Plans (IDPs) in conjunction with their supervisor each year IDPs list areas for development and training for each staff member for the coming calendar year Target: 33% of staff will have IDPs developed by February 28, 2003 Results: 39% of staff had IDPs by February 28, 2003

25. 25 Financial Perspective

26. 26 Financial Perspective

27. 27 F1: Change in “Unit Cost of Information” – Number of Units Used Use of information units has increased 35% in the past four years A unit of information may be a search, a document supplied, a customer trained, an article downloaded, or a visit to our web site The electronic resources are heavily used and their use increases each year

28. 28 F1: Change in “Unit Cost of Information” – Unit Costs FY00–FY03 Target: Maintain or reduce the cost of information provided to NIH staff The unit cost of information is calculated by dividing the cost of the information into the number of information units used The unit cost of information has remained fairly steady over the past 4 years E-resources are more expensive than print resources; however, they are used more heavily thereby keeping the unit cost of information steady

29. 29 F2-6a: Change in Unit Cost The unit cost for provide copies of publications is lower in FY04. In FY03 276,000 copies of publications were supplied. As more documents are delivered as e-documents, costs are reduced. The unit cost for self-service copiers rose in FY04. In FY03 there were 24,000 visits to the self-service copiers. Use of copiers has decreased and hence the unit cost for the service has increased.

30. 30 F2-6a: Change in Unit Cost (cont.) The unit cost for translate documents has risen over the past 4 years. The costs are largely based on salaries and contracting costs. 755 items were translated in FY04. The unit cost for provide primary library services has risen over the past 4 years. This service is growing. Costs are primarily based on salaries and purchase of print and e-resources. The cost of print and electronic resources rises 10- 20% each year, and the Library is increasing the amount spent on e- resources by 20% each year. There were 9.6 million units of information provided in FY04. The unit cost for provide custom research assistance has not changed since last year. Seventeen custom projects were undertaken in FY04.

31. 31 Conclusions

32. 32 Accomplishments for FY03 Customers are very satisfied with the Library’s resources and services. The website has been redesigned in content management software, a new brand has been developed for the Library, and new print materials have been designed. These materials will be introduced to the NIH staff on February 10, 2004. Full-Text Plus! software that will enable users to link easily from databases to journal articles to training and more has been implemented. Relais II has been implemented, which has streamlined the process used to fill document requests, and a web-based document request form is available to customers. The “Unit Cost of Information” remains steady as we add new electronic resources to the collection.

33. 33 Initiatives for FY04 We are in the process of developing a web-based survey, in conjunction with the NLM, to assess customer satisfaction with the informationist service. We will be conducting either the bi-annual user satisfaction telephone survey or a user needs assessment in FY04, and we will use the scorecard to gauge customer satisfaction with provide copies of publications and reference service provided at the information desk. The Parklawn Health Library will become a branch of the NIH Library. In FY04, MetaLib, software that will enable searching across many e-resources, will be implemented. The Web of Science software will be updated to the Web of Knowledge. Development of an in-house training curriculum to ensure all staff are knowledgeable about Library services and to retrain staff for new work will continue. To maintain the unit cost of information, resources and services will continue to be selected based on their usefulness to NIH staff and cost per use will be a major factor in the selection and retention of electronic resources.

34. 34 Appendix Customer Perspective C1a: Percent of current print journals in electronic format……………..…38 C1b: Percentage of license agreements that cover all NIH……………....39 C1c: Percentage of document requests filled in-house…..……………….40 C1d: Number of items translated by government staff and contractors…41 C2a: Number of web-based instruction modules………………………..…43 C2b: Training attendance by type……………………………………………44 C2c: Training evaluation scores…………………………………………..…45 C3a: Customer Scorecard ratings for translation service and mediated database search service..………………………………….…..48 C3c: Digital reference service evaluation ratings…………………………..54 C4: Customer segmentation data……………………………………..……..56 Internal Business Process Perspective IB1: Number and type of special projects contracted for…………..…..…63 IB2: Number of new IT systems implemented…………………………..…65 IB3a: Percent of document requests filled in-house in 3 business days……………………………………………………………………….….69 IB3b: Percentage of documents translated by library staff within 3 business days…………………….…….……………………..…..70 IB3c: Percent of tutorial requests responded to within 1 business day………………………………………………….….….71

35. 35 Appendix (cont.) Learning and Growth Perspective LG1a: Percentage of staff members with IDPs completed by February 28 each fiscal year……………………………………...........…74 LG1b: Percentage of staff completing 80% of activities listed in IDP.......................................................................................................75 LG1c: Percentage of training modules completed on schedule…………76 Financial Perspective F1: Change in unit cost of information…………….……………………….79 F2a: Change in unit cost of custom services…………………………..….82 F2b: No. of contracts/IAGs……………………………………………….….82 F3a: Change in unit cost of providing copies of publications…………….82 F3b: No. of document requests filled……………………………………….82 F4a: Change in unit cost of translating services…………………….…….83 F4b: No. of documents translated…………………………………….…….83 F5a: Change in unit cost of providing primary library services……….….83 F5b: No. of units of information provided…………………………….…….83 F6a: Change in unit cost of providing self-service copiers…………...….83 F6b: No. of visits to self-service copy center………………………..…….83

36. 36 Customer Perspective

37. 37 Customer Perspective

38. 38 C1a: Percent of current print journals in electronic format Target: Provide 100% of current journals in electronic format Eighty-one percent of the journals are now available in electronic format (as compared to 75% in FY02 and 52% in FY01)

39. 39 C1b: Percentage of license agreements that cover all NIH Target: Negotiate all electronic resource license agreements for access by all of NIH Currently 62% of e- license agreements give all NIH staff access (as compared to 53% in FY02 and 45% in FY01)

40. 40 C1c: Percentage of document requests filled in-house Target: Maintain percentage of document requests filled in-house or at NLM to keep costs down and turn around time fast Results: Percentage filled in- house in FY03 was 82% and in FY02 was 84% The large number of requests that can be filled in- house reflects the careful tailoring of the collection to the needs of the NIH staff

41. 41 C1d: Number of items translated by government staff and contractors The total number of items translated in FY03 was 755 items The number of words translated has increased 44% over the past 6 years To ensure items are translated in a timely manner increasingly, items are given to contractors to translate

42. 42 Customer Perspective (cont.)

43. 43 C2a: Number of web-based instruction modules Target: Develop web-based instruction modules for all training classes, if feasible We have found that realistically only about half of our classes can be made into web-based modules. Library staff currently are busy updating all modules with new NIH Library branding

44. 44 C2b: Training attendance by type Customers have clearly indicated their preference for web- based training which is available 24/7 Nevertheless, attendance at classes remains steady Class attendance numbers are used to determine which classes to offer

45. 45 C2c: Training evaluation scores Classes are provided to teach NIH staff effective use of electronic resources and bibliographic management software In FY03, 75% of attendees (451/599) completed an evaluation form designed to assess satisfaction with the training classes Customers were asked to rate their satisfaction on a 5 point scale with 1 being poor and 5 being excellent The classroom was upgraded at the beginning of FY03 and the rating for the classroom rose from 4.2 to 4.6

46. 46 C2c: Training evaluation scores

47. 47 Customer Perspective (cont.) Following are the results of the survey of customers satisfaction with three services: Translation Services Mediated database searching Digital reference service Evaluations were transaction based All services received high scores

48. 48 C3a: Customer Scorecard ratings for translation service Evaluation surveys were distributed to customers who used the translation service in FY02 and FY03 Customers were asked to rate satisfaction and importance of the service on a 10 point scale with 1 being unsatisfactory and 10 being outstanding Customers were highly satisfied with all aspects of the translation service. In addition, customers felt that timeliness was much more important than cost Survey distributionFY02FY03 No. of surveys distributed88390 No. of respondents3770 Response rate42%18%

49. 49 C3a: Customer Scorecard ratings for translation service – Satisfaction Ratings * FY03 availability ratings are significantly higher (p<.05) than FY02 ratings

50. 50 C3a: Customer Scorecard ratings for translation service – Importance Ratings

51. 51 C3a: Customer Scorecard ratings for mediated database searching service Mediated database searching service was evaluated from mid-October to mid-December of 2003 using paper or electronic survey forms Forty surveys were distributed and 14 were returned for a return rate of 35% The surveys asked users of the service to rate customer service and search results on a 10 point scale with 1 being unsatisfactory and 10 being outstanding Customers were very satisfied with the service

52. 52 C3a: Customer Scorecard ratings for mediated database searching service – Customer Service

53. 53 C3a: Customer Scorecard ratings for mediated database searching service – Search Results

54. 54 C3c: Digital reference service evaluation ratings The Library has offered a digital reference service for the past two years Digital reference combines chat and the ability to push web pages to customers in response to a reference question Customers are given the option of filling out an evaluation form at the end of each transaction In FY02 37% of customers responded to the survey and in FY03 the response rate was 39% Customers found the service easy to use and would use it again. Survey questions will be revised in FY04 to give a more rounded view of the customer’s satisfaction with the service.

55. 55 C3c: Digital reference service evaluation ratings

56. 56 Customer Perspective (cont.)

57. 57 C4: Customer segmentation data Customer segmentation data is presented for four services: Provide copies of publications Provide self-service copiers Translate documents Provide custom research assistance As in past years, customers are spread across all IC’s Heavier use of services by larger IC’s and groups most directly engaged in patient care and biomedical research The Library serves not only NIH but also FDA/CBER on the Bethesda campus and provides document delivery to several groups including NIST and CDC.

58. 58 C4: Customer Segmentation for Provide Copies of Publications

59. 59 C4: Customer Segmentation for Provide Self-Service Copiers

60. 60 C4: Customer Segmentation for Translate Documents

61. 61 C4: Customer Segmentation for Provide Custom Research Assistance

62. 62 Internal Business Perspective

63. 63 Internal Business Perspective

64. 64 IB1: Number and type of special projects contracted for Target for FY04 is to hire more informationists and increase the number of informationist assignments to 8 Custom ServicesFY02FY03 Collection Related67 Information Related Bibliography, database, web page75 Informationist assignments35 Totals1617

65. 65 Internal Business Perspective (cont.)

66. 66 IB2: Number of new IT systems implemented There were four areas of concentration in FY03: HHS Library Consortium SFX or Full-Text Plus! Relais Content Management Software The major areas of concentration for FY04 are: Continue with the development of the HHS Consortium, Full- Text Plus!, Relais, and CMS Implement MetaLib Enable wireless access in the Library

67. 67 New IT systems implemented in FY03 HHS Library Consortium –In FY03 the Consortium jointly purchased the Nature journals, Science, and Science Express at a considerable savings to NIH Full-Text Plus! – This software, which links various services and resources available from the Library, was implemented for the online journals, the Web of Science and Ovid databases, the new online document request form, and other resources in FY03 Relais – In FY03, phase II of the Relais system was implemented with the integration of the web-based request form and the Web of Science requests. Through Relais the processing of document requests has been considerably streamlined. A process that used to have eight steps now has only three with a turn around time of 1- 3 business days Content Management Software has been implemented for the Library’s Intranet and public websites resulting in a diversification of responsibility for the sites. The redesigned public website will be unveiled on February 10, 2004

68. 68 Internal Business Perspective (cont.)

69. 69 IB3a: Percent of document requests filled in–house in 3 business days Target: Fill 96% of requests in 3 business days Results: Average fill rate for FY03 was 98% and for FY03 was 97%

70. 70 IB3b: Percentage of documents translated by library staff within 3 business days Target: Translate document requests in 3 business days Results: In FY03 an average 55% of documents were translated in 3 business days. In FY04 the translation speed was measured for Library translators and contractors separately. In FY04 Library translators translated 55% of documents in 3 days and contractors translated 26% of documents in 3 days. Efforts to improve the turn around time for translating documents are complicated by: Translation requests are increasing and the current staff are already working as efficiently as possible The more translations that are contracted out, the more time the Library translation staff must spend checking contracted translations thereby reducing the time they have to translate

71. 71 IB3c: Percent of tutorial requests responded to within 1 business day

72. 72 Learning and Growth Perspective

73. 73 Learning and Growth Perspective

74. 74 LG1a: Percentage of staff members with IDPs completed by February 28 each fiscal year Staff are asked to develop Individual Development Plans (IDPs) in conjunction with their supervisor each year IDPs list areas for development and training for each staff member for the coming calendar year Target: 33% of staff will have IDPs developed by February 28, 2003 Results: 39% of staff had IDPs by February 28, 2003

75. 75 LG1b: Percentage of staff completing 80% of activities listed in IDP Target: Eighty percent of staff will have completed 80% of IDP objectives by Dec. 30 Result: 88% of staff with plans in place in February completed 80% of their objectives

76. 76 LG1c: Percentage of training modules completed on schedule Training Project Team established in FY03 Projects for FY04 are: Survey Library staff to assess the state of learning in the NIH Library Conduct needs assessment Develop training plan

77. 77 Financial Perspective

78. 78 Financial Perspective

79. 79 F1: Change in “Unit Cost of Information” – Number of Units Used Use of information units has increased 35% in the past four years A unit of information may be a search, a document supplied, a customer trained, an article downloaded, or a visit to our web site The electronic resources are heavily used and their use increases each year

80. 80 F1: Change in “Unit Cost of Information” – Unit Cost FY00–FY03 Target: Maintain or reduce the cost of information provided to NIH staff The unit cost of information is calculated by dividing the cost of the information into the number of information units used The unit cost of information has remained fairly steady over the past 4 years E-resources are more expensive than print resources; however, they are used more heavily thereby keeping the unit cost of information steady

81. 81 F1: Calculations for the “Unit Cost of Information”

82. 82 F2-6a: Change in Unit Cost The unit cost for provide copies of publications is lower in FY04. In FY03 276,000 copies of publications were supplied. As more documents are delivered as e-documents, costs are reduced. The unit cost for self-service copiers rose in FY04. In FY03 there were 24,000 visits to the self-service copiers. Use of copiers has decreased and hence the unit cost for the service has increased.

83. 83 F2-6a: Change in Unit Cost (cont.) The unit cost for translate documents has risen over the past 4 years. The costs are largely based on salaries and contracting costs. 755 items were translated in FY04. The unit cost for provide primary library services has risen over the past 4 years. This service is growing. Costs are primarily based on salaries and purchase of print and e-resources. The cost of print and electronic resources rises 10- 20% each year, and the Library is increasing the amount spent on e- resources by 20% each year. There were 9.6 million units of information provided in FY04. The unit cost for provide custom research assistance has not changed since last year. Seventeen custom projects were undertaken in FY04.

84. 84 Conclusions

85. 85 Accomplishments for FY03 Customers are very satisfied with the Library’s resources and services. The website has been redesigned in content management software, a new brand has been developed for the Library, and new print materials have been designed. These materials will be introduced to the NIH staff on February 10, 2004. Full-Text Plus! software that will enable users to link easily from databases to journal articles to training and more has been implemented. Relais II has been implemented, which has streamlined the process used to fill document requests, and a web-based document request form is available to customers. The “Unit Cost of Information” remains steady as we add new electronic resources to the collection.

86. 86 Initiatives for FY04 We are in the process of developing a web-based survey, in conjunction with the NLM, to assess customer satisfaction with the informationist service. We will be conducting either the bi-annual user satisfaction telephone survey or a user needs assessment in FY04, and we will use the scorecard to gauge customer satisfaction with provide copies of publications and reference service provided at the information desk. The Parklawn Health Library will become a branch of the NIH Library. In FY04, MetaLib, software that will enable searching across many e-resources, will be implemented. The Web of Science software will be updated to the Web of Knowledge. Development of an in-house training curriculum to ensure all staff are knowledgeable about Library services and to retrain staff for new work will continue. To maintain the unit cost of information, resources and services will continue to be selected based on their usefulness to NIH staff and cost per use will be a major factor in the selection and retention of electronic resources.

87. 87 The End