1 FY02 ASA Presentation Provide Library Services Presented by: Susan Whitmore Chief, Information & Education Services Section NIH Library Office of Research Services National Institutes of Health 18 November 2002

2 Table of Contents Main Presentation ASA Template ……………………………….…………………………….….4 Customer Perspective……………………….…………………………….….5 Customer Segmentation …………………….…………………...…………..……5 Customer Satisfaction……………………….…………………………………….12 Internal Business Process Perspective …………………………………….…….20 Conclusions from Discrete Services Deployment Flowcharts……….…..……21 Process Measures…………………………………………………………….…...22 Learning and Growth Perspective ……………………………………………..….32 Conclusions from Turnover, Sick Leave, Awards, EEO/ER/ADR Data….…..33 Analysis of Readiness Conclusions…………………………………………..….34 Financial Perspective……………………………………………………..…. 36 Unit Cost……………………………………………………………………….……36 Asset Utilization……………………………………………………………….……39 Unique Financial Measures..……………………………………………….……..43 Conclusions and Recommendations………………………………….……. 50 Conclusions from FY02 ASA..……………………………………………….……51 Recommendations………………………………………………………….………52

3 Table of Contents Appendix Page 2 of the ASA Template Customer segmentation graphs Customer satisfaction graphs Block diagram Process charts/Deployment flowcharts Internal Business Performance Measures and Control Charts Learning and Growth data table Analysis of Readiness Information Asset utilization data Unit Cost of Information data

4

5 Customer Perspective --Customer Segmentation

6 Telephone Survey Telephone survey of 400 randomly chosen Library users conducted in 2000 and 2002 Included questions about primary library services, providing copies of publications, and self-service photocopy service Customer segmentation data reveals customer’s primary work and position Most respondents engaged in research Most respondents identified themselves as post doctoral fellow, staff scientist, administrator, senior investigator, or research fellow

7 How Telephone Survey Respondents Spend Their Time

8 Positions Held by Telephone Respondents

9 Customer segmentation data by IC available for providing copies of publications, self-service copiers, translate documents and custom service Customers spread across all IC’s Heavier use of services by larger groups and IC’s engaged directly in biomedical research, ex., NCI, NIAD, NICHD, NINDS, NHLBI Customer Segmentation

10 Customer Segmentation continued Less use by smaller groups and IC’s whose work is administrative or a support service, ex., OD, ORS, FIC, CIT The Library serves not only NIH but also FDA/CBER on the Bethesda campus and provides document delivery services to the National Institute of Standards and Technology (NIST), Centers for Disease Control (CDC), and NIEHS Customer segmentation data for self-service photocopy, translations, and custom services may be found in the Appendix

11 Customer Segmentation for Provide Copies of Publications

12 Customer Perspective --Customer Satisfaction

13 The 2002 Telephone Survey revealed that 97% of NIH staff give a positive rating to Library services overall Customer satisfaction levels increased slightly in two years – in FY00 94% of surveyed staff gave a positive rating to Library services overall All services surveyed received a very positive rating Customer Satisfaction

14 Percentage of NIH Staff Who Gave a Positive Rating to the Quality of Library Services in FY02

15 Customer Satisfaction for Provide Translations Thirty-three percent response rate for evaluations Both product and customer service ratings exceeded the ORS average ratings Many customers commented that the translation service is excellent Timeliness was more important to the customers than cost Customers were given realistic expectations with regard to turnaround time but some would like documents more quickly

16 Customer Satisfaction for Translation Product

17 Customer Satisfaction for Translation Customer Service

18 Unique Customer Measures – Maintain Customer Satisfaction for Seminars Seminars are provided to teach NIH staff how to effectively use electronic resources and bibliographic management software 77% of attendees (417/544) completed an evaluation form designed to assess satisfaction with the training classes Choices were 5=excellent, 4=very good, 3=good, 2=fair, 1=poor Control charts indicate no signals (Control charts may be found in the appendix)

19 Seminar Evaluation Results for FY02

20 Internal Business Process Perspective --Deployment Flowcharts

21 Our Service Group completed deployment flowcharts for all five discrete services (which may be found in the appendix) Most discrete services are complex so very little detail could be indicated on the flowcharts Nothing new was learned from completing the flowcharts Conclusions from Discrete Services Deployment Flowcharts

22 Internal Business Process -- Measures for each Discrete Service

23 DS1: Custom Services Objective: Increase number of custom service projects Measure: Percent increase in number of contracts established each year Results: Baseline data for FY02 – 17 projects

24 DS2: Provide Copies of Publications Objective: Maintain or reduce turn around time for filling document requests Measure: Percent of requests filled in 3 business days Goal: Fill 96% of requests in 3 business days Results: Average fill rate of 97% for FY02 is above the goal. Goal was not met in August due to two staff members on maternity leave and other staff on vacation Control chart may be found in the Appendix

25 DS3: Translate Documents Objective: Maintain or decrease the turn around time for translating documents Measure: Percent of documents translated in 3 business days, protocols in 10 business days, articles in 30 business days Results: On average 55% of documents, 52% of protocols, and 47% of articles were translated in the required time period Priority is given to translating documents which include diplomas, transcripts, and personal documents which the FIC needs for appointments of research fellows. Protocols are primarily from English to another language and must be contracted out and reviewed in- house. The following control chart for document translations shows a signal for July when the translators were on vacation. Other control charts are in the Appendix

26 Translation Requests Control Chart for Documents Note: Percentage of documents translated within goal of 3 business days is depicted by month

27 However Translation Requests Are Increasing The number of words translated has increased 35% over the past year Translations for languages outside translator’s areas have always been contracted out. To keep up with demand, increasingly translations for in- house language capability are contracted out Contracted out translations take more time. Next year, the time it takes to complete a translation by in-house staff versus a contractor will be tracked separately If enough translation requests for one or several related languages are consistently received, this would justify hiring another translator

28 DS4: Provide Primary Library Services Objective: Provide 100% of current journals in electronic format Measure: Percent of current journals subscribed to electronically Once almost all (93%) of the most used 350 journals were in electronic format, we changed out goal to have all journals in e-format. Currently, 75% of all journals are in electronic format

29 Additional Internal Business Measures for Provide Library Service located in the Appendix --E-license agreements to include all of NIH --Ready access to training --Quick response to requests for tutorials

30 DS5: Provide Self-Service Copiers Objective: Provide adequate number of photocopy machines for users Measure: Ratio of photocopy machines to visits Six copy machines were made available for customer throughout FY02. There is no waiting for a machine, therefore, the number of copiers is adequate

31 However Self-Service Use is Declining Use of self-service copying continues to decline as the Library obtains additional e- resources Hours of service were reduced in FY01 due to declining use The number of copiers and hours of service will be evaluated periodically and reduced as warranted

32 Learning and Growth Perspective

33 An average of 4.5 days sick leave per employee is above average for ORS and reflects the three staff who were on maternity leave in FY02 Eight awards for every 10 employees is slightly below average for ORS. Two factors limit the number of awards given per person: The AFGE Union contract mandates that 70% of our award pool be given as performance awards As a team-based organization, we have chosen to use the other 30% of the award pool to reward teams. The turnover and EEO/ER/ADR data appear fairly average for ORS Conclusions from Turnover, Sick Leave, Awards, EEO/ER/ADR Data

34 The way in which information is being provided and accessed by customers is changing: Customers are increasingly using e-resources that they access from their desktop computers Customers are answering many of their own easier information questions Customers are increasingly accessing our services through our web site The services that aid customers in information identification, acquisition, and management are changing: Library staff need to provide services electronically such as the digital reference service Library staff need to reach out to customers in their offices and labs as the Informationists do Services that make it possible for users to search across a number of e-resources and link directly to online journals are needed such as MetaLib and SFX The Library’s web site continues to grow and be essential to service provision. The content needs to be dynamic and up-to-date Analysis of Readiness - Conclusions

35 Analysis of Readiness – Conclusions -continued Library staff need to be retrained to engage in new tasks Librarians need to learn in-depth searching techniques for animal alternatives and evidence-based medicine. They need biomedical subject expertise to offer customized services to researcher and clinicians in their offices and labs Library Technicians must learn new skills sets as the old work diminishes and new work is created The Library is identifying training programs and/or developing in- house programs to address the new skills that are needed by staff We are very aware of the changes in information resources and services and are well poised to take advantage of these changes A more thorough analysis of the Library’s readiness may be found in the Appendix

36 Financial Perspective --Unit Cost Measures

37 Unit Cost The average unit cost has decreased over the past two years for: Copies of publications (-2%/yr) Self-service copiers (-14%/yr) As many users are accessing information through the Library’s e-resources, these services have declined in use and we anticipate that they will continue to decline in use The cost of these services is primarily based on salaries, supplies and equipment. As the use of the service declines, the unit cost of the service will decline also

38 Unit Cost continued The average unit cost has increased over the past two years for: Translate documents (+13%/yr) Primary library services (+7%/yr) Custom research assistance (+15%/yr) These services are growing. Costs are primarily based on salaries and purchase of print and electronic resources The cost of print and electronic resources rises 10-20% each year. The Library is increasing the amount spent on e- resources by 20% each year

39 Financial Perspective --Asset Utilization Measures

40 Asset Utilization for Discrete Services DS1: Provide Custom Research Assistance – no data DS2: Provide Copies of Publications – 98% DS3: Translate Documents – 74% DS4: Provide Primary Library Services – Instruction Program – 74% DS5: Provide Self-Service Copiers – 63% Descriptions of how the asset utilization percentages were calculated may be found in the Appendix and on the following slide

41 DS4: Provide Primary Library Services Primary library services are composed of instruction, reference, mediated searching, and the print and electronic collection Asset utilization data for part of the instruction program, seminars held in the Library on Thursdays, was gathered The maximum number of potential class attendees were compared to the actual number of attendees 74% asset utilization = No. of actual class attendees (544)/No. of potential attendees (741) X 100 Promotion of the seminars early in the year increased attendance and we increased the number of workstations from six to 20 during the summer

42 Instruction Program Asset Utilization

43 Unique Financial Measures --Unit Cost of Information --Percent of Library Budget Devoted to E-resources --Minimize Training Costs

44 Units of Information Use of information units has increased 26% in two years A unit of information may be a search, a document supplied, a customer trained, an article downloaded, or a visit to our web site Because e-resources are heavily used, more information is available to customers at a reduced cost per unit of information

45 Unit Cost of Information Objective: Maintain or reduce the cost of information provide to NIH staff Measure: Change in unit cost of information The need to include rent and overhead in the FY01 budget increased the unit cost of information E-resources are more expensive than print resources; however, they are used more heavily. Thus the unit cost of information for FY02 is the same as for FY00 See the Appendix for the definition of how the unit cost is determined

46 Percent of Library Budget Devoted to E-resources Objective: Increase availability of highly used e- resources Measure: Percent of library budget devoted to electronic versus print resources Result: The percent of the library budget devoted to electronic resources has increased 1,600% in the past six years. Percent of budget devoted to e- resources has risen 21% in the past year

47 Percent of Library Collection Budget Devoted to Electronic Resources

48 Electronic vs. Print Journals The percentage of journals in electronic format has increased 29% However, many resources are received as print + electronic and these resources are still reflected in the print budget

49 Minimize Training Costs Objective: Minimize training costs by providing users with alternate means of training Measure: Percent of people trained in seminars or via tutorials versus the number of people using web-based training Result: Customers have clearly indicated their preference for web-based training which is available at any time

50 Conclusions and Recommendations

51 Conclusions from FY02 ASA Our customers are very satisfied with the Library’s resources, services, and staff Internal business processes are for the most part efficient and cost effective There will be a growing number of electronic resources and services and more customized services in the future and there need to be concomitant changes in staff skill sets Use of resources and services, particularly e-resources, continues to grow. We will continue to convert from a primarily print to an electronic library. While electronic resources are expensive, they are utilized much more heavily than the print collection thereby keeping the unit cost of information reasonable

52 Recommendations - Specific Develop means to determine utilization of assets for Provide Custom Research Assistance by March 2003 Develop an evaluation form for Provide Custom Research Assistance projects by February Work with National Library of Medicine staff to evaluate Informationist program in 2003 and 2004 Continue to collect data for Internal Business Measures for Provide Self-Service Copiers and construct a control chart for the ASA in November 2004 Separately collect data for the time it takes to complete translations by in-house staff versus contractors to determine how service speed can be improved

53 Continue with the redesign of the web site as recommended in last year’s ASA. Projected date for roll out January 15, Implement content management software for web site to ensure up-to- date and dynamic content. Build the databases needed to provide content. Projected date for completion January 15, Incorporate software called MetaLib and SFX into our web site This software will enable users to search across e-resources and link directly from databases to online journals. This recommendation partially replaces a similar one from last year’s ASA. Implementation began September Completion of initial phase of implementation March Develop an in-house training curriculum to ensure all staff are knowledgeable about Library services and to retrain staff for new work. Development will begin in January Full implementation will take two years. Recommendations - Broad

54 Appendix

55 ASA Template – Page 2

56

57

58 Customer Segmentation Graphs --Provide Self-Service Photocopy --Translate Documents ---Provide Custom Services

59 Provide Self-Service Photocopy

60 Translate Documents

61 Custom Services FY02

62 Customer Satisfaction Graphs --Translate Documents -- Seminar Evaluation Control Charts for FY02 – no signals --Unique Customer Measure – Acquire Electronic and Print Resources Need by Users

63 Translate Documents Scatter Diagram

64 Seminar Evaluation Control Chart - Clarity of instructor’s presentation Note: Monthly mean ratings are depicted based on a rating scale ranging from (1) Poor to (5) Excellent.

65 Seminar Evaluation Control Chart – Instructor’s knowledge of subject Note: Monthly mean ratings are depicted based on a rating scale ranging from (1) Poor to (5) Excellent.

66 Seminar Evaluation Control Chart – Instructor’s responsiveness to questions Note: Monthly mean ratings are depicted based on a rating scale ranging from (1) Poor to (5) Excellent.

67 Seminar Evaluation Control Chart – Quality of Handouts mNote: Monthly mean ratings are depicted based on a rating scale ranging from (1) Poor to (5) Excellent.

68 Seminar Evaluation Control Chart – Level to which course objectives were met Note: Monthly mean ratings are depicted based on a rating scale ranging from (1) Poor to (5) Excellent.

69 Seminar Evaluation Control Chart – Quality of classroom arrangements Note: Monthly mean ratings are depicted based on a rating scale ranging from (1) Poor to (5) Excellent.

70 Unique Customer Measure – Acquire Electronic and Print Resources Needed by Users Acquired 42% of print and electronic resources requested by customers in FY02 Reasons for not acquiring resources are: Low usage Cost Unavailable in electronic format Lack of suitable license agreement

71 Internal Business Performance Measures --Block diagram and process charts

72

73

74

75

76

77

78 Internal Business Performance Measures --Provide Copies of Publications -Control Chart

79 Provide Copies of Publications - Control Chart Note: Percentage of requests filled within goal of 3 business days is depicted by month. Goal = 96%

80 Additional Internal Business Performance Measures Following are three additional Internal Business Performance Measures for DS4: Provide Primary Library Services --Negotiate e-license agreements that include all of NIH --Balance need for web-based training with hands- on training --Respond to tutorial requests in one business day

81 DS4: E-Resource License Agreements Objective: Negotiate e- license agreements that include all of NIH Measure: Percent of license agreements that include all of NIH Results: At the end of FY02, 53% of e-resource license agreements included all of NIH – an increase in 8% over last year

82 DS4: Web-based Training Objective: Balance need for web-based training with hands-on classes Measure: Percent of courses that are web-based Results: 37% of the 51 classes offered are web- based 61% of customers prefer web-based training but we will continue to provide hands-on classes as long as attendance indicates that customers want this type of training

83 DS4: Tutorial Request Response Objective: Respond to tutorial requests within one business day Measure: Percent of requestors contacted within one business day Results: 100% of requestors contacted within one business day in FY02

84 Internal Business Perspective --Control charts for Translate Documents showing percentage of protocols translated in 10 business days and articles in 30 business days – no signals

85 Translate Protocols in 10 Business Days Note: Percentage of protocols translated within goal of 10 business days is depicted by month

86 Translate Articles in 30 Business Days Note: Percentage of articles translated within goal of 30 business days is depicted by month

87 Internal Business Perspective --Provide Self-Service Copiers

88 DS5: Provide Self-Service Copiers Ratio of photocopy machines to visits is shown on the accompanying chart There were six copiers available in FY02 Customers do not have to wait for a copy machine, therefore, the number of copiers is adequate

89 Learning and Growth Perspective --Learning and Growth Data Table

90 Learning and Growth Data Table About 4.5 days sick leave per employee About eight awards for every 10 employees 5% employee turnover 1 ER case out of 50 employees 1 ADR case out of 50 employees

91 Learning and Growth Perspective --Analysis of Readiness Information

92 Several changes in service and resources in the Library will require staff with a different mix of skills Fewer document requests will be received as more e- resources are made available and there will be less work for the document delivery staff Implementation of NED results in less work for circulation staff More books and journals will be obtained in e-format which will require less clerical support Customers are answering many of their reference questions using e-resources and the types of reference questions we receive at the reference desk are less difficult and can be answered by a well trained paraprofessional. Information services need to be customized to users specific needs. Staff need to be trained to fully use e-resources effectively.

93 Several changes in service and resources in the Library will require staff with a different mix of skills - continued Customers are using the Library’s resources through their computers instead of walking into the Library. Library staff need to offer services that reach out to NIH staff in their labs and offices as the Informationist service does Librarians will be performing more custom services which will require more knowledge of biomedicine. Librarians will be asked to conduct more complicated database searches for animal alternatives and evidence-based medicine More translations are being requested Systems office staff will be needed to develop and program databases to track and make e-resources available and accessible

94 Changes in the way information is accessed will require a change in the way information is provided Customers are using the Library’s resources through their computers instead of walking into the Library. More services, such as reference service and instruction, need to be offered to staff electronically The growth in the number of e-resources requires the acquisition of software and systems that link databases to online journals and that enable customers to search efficiently across resources Users are requesting additional e-resources which are often more costly than print resources Specialized databases need to be created to organize and track e-resources and to populate the web site

95 Changes in service and resources will require that: Library technicians working in document delivery, circulation, and collection organization & maintenance will have less work and will need to develop new skill sets in order to remain employed in the Library Library technicians trained to respond to customer requests at the information desk will free the professional librarians to provide custom services The Library develop training programs to ensure that staff develop needed skills Librarians take classes from outside sources to develop more in-depth skills in database searching and knowledge of biomedicine Systems Office staff take training in database management and programming Content management software be implemented for the web site. This software will make it easier to update and maintain web site and free librarians for other activities

96 To ensure that staff are ready to take on new assignments, the Library is: Continuing to develop as a “Learning Organization” Developing in-house programs such as the Reference Assistants training program which is designed to train document delivery and circulation staff to work at the information desk Developing a training program for Informationists who will reach out to NIH researchers and clinicians to supply customized information assistance Appointing a Training Coordinator to organize training programs and develop ways to evaluate effectiveness of training Identifying courses and training programs that will prepare staff for new work Using COTA’s as appropriate to develop in-house talent Continuing to use cross-functional teams to enable staff to learn new skills and take on new assignments Monitoring translation requests and hiring more staff with the appropriate language skills as the situation warrants

97 Implications of Not Obtaining Right Mix of Skills or Materials The way in which information is being delivered has changed. If the Library is not able to provide e-resources to customers, the customers will be impeded in their work and ultimately will go elsewhere for the needed information Library technician work is decreasing, staff who are not retrained for new work will no longer be employed by the Library Custom services that reach out to customers to solve their unique information needs will be in demand. Professional librarians need to be trained to provide these services and need to be freed from lower level tasks so that they have time to provide them. If the Library does not provide these new services, customers will hire contractors to do the work. If professional librarians continue to provide services that can be transferred to well trained library technicians, the Library will not be operating in a cost efficient manner

98 Financial Perspective Asset Utilization --Provide Custom Research Assistance --Provide Copies of Publications --Translate Documents --Provide Self-Service Copiers

99 DS1: Provide Custom Research Assistance Asset utilization data has not been collected for this service We will research how to collect this data for next year

100 DS2: Provide Copies of Publications The maximum number of hours that could have been spent providing publications was compared to the actual number of hours spent on this service in FY02 Included are government and contract staff hours 98% asset utilization = 37,435 actual hours/38,215 potential X 100 In FY03 we plan to keep a log to sample the amount of time government and contract staff spend on this discrete service

101 DS3: Translate Documents The maximum number of hours that the translators could ideally devote to translating were compared to the number of hours they actually spent translating 74% asset utilization = translators spent 2,723 hours translating/3,680 hours that ideally could be spent translating X 100 Hours not spent translating are spent attending meetings, contracting out translations, doing clerical work, and developing an ORS/ASA report In June, the Library’s labor contract was used to assume much of the clerical work and reduce the clerical work burden on the translators An asset utilization chart follows

102 Translation Asset Utilization Graph

103 DS5: Provide Self-Service Copiers The actual number of copies made on the six copies in the self- service copy center were compared to the number of copies that could have been made, based on the normal rate at which customers make photocopies and the total hours the center is open 63% asset utilization = 1,498,192/2,384,100 X 100 As shown in the internal business perspective, use of the self- service copy center is declining as customers use the e- resources Use of the self-service copy center will be monitored and hours reduced or the number of copy machines reduced as warranted

104 Financial Perspective --Unit Cost of Information Data

105

106 The End