1. Calculating ROI in Special Libraries and Information Centers: Consequences of NOT Having an Information Center José-Marie Griffiths, Ph.D. Sarah E. Aerni University of North Carolina at Chapel Hill June 2, 2007 Special Library Association Annual Conference

2. Topics Covered in Workshop Return-on-Investment (ROI) defined Two examples of ROI projects –ROI of entire special library –ROI of access to journal collections: print and electronic ROI methods Communicating results

3. Role of Evaluation Planning Resource allocation and management Justification of new or existing resources Advocacy, marketing and public relations

4. Conceptual Framework for Evaluation

5. Object of Evaluation Library Function Service Activity Resource

6. UserOrganization Industry/ Sector Society Library Measurement Perspectives Conceptual Framework for Library Economic Measures

7. Industry/ Sector Inputs (Resources) Amount Cost Attributes Measurement Perspectives Specific Measures Conceptual Framework for Library Economic Measures UserOrganization Industry/ Sector Society Library

8. Inputs (Resources) Amount Cost Attributes Outputs (Products/Services) Amount Attributes Quality Timeliness Availability Accessibility Measurement Perspectives Specific Measures Conceptual Framework for Library Economic Measures UserOrganization Industry/ Sector Society Library

9. Inputs (Resources) Outputs (Products/ Services) Usage (Use & Nonuse) Amount Factors affecting use/nonuse Ease/cost of use (price paid) Available alternatives Purpose of use Importance of and satisfaction with attributes of output Awareness Measurement Perspectives Specific Measures Conceptual Framework for Library Economic Measures UserOrganization Industry/ Sector Society Library

10. Inputs (Resources) Outputs (Products/ Services) Usage (Use & Nonuse) Outcomes (Consequences of Use and Nonuse) Time saved Improved learning Improved productivity Improved quality of work Improved timeliness of work Value derived Effects on organization goals Higher order effects Measurement Perspectives Specific Measures Conceptual Framework for Library Economic Measures UserOrganization Industry/ Sector Society Library

11. Inputs (Resources) Outputs (Products/ Services) Usage (Use & Nonuse) Outcomes (Consequences of Information) Domain (Environmental Characteristics) Target population User/nonuser population characteristics User/nonuser needs/requirements Externalities Measurement Perspectives Specific Measures Conceptual Framework for Library Economic Measures UserOrganization Industry/ Sector Society Library

12. Derived Measures Performance Specific Measures Measurement Perspectives Conceptual Framework for Library Economic Measures Inputs (Resources) Outputs (Products/ Services) Usage (Use & Nonuse) Outcomes (Consequences of Information) Domain LibraryUserIndustry/ Sector OrganizationSociety

13. Derived Measures Performance Effectiveness Specific Measures Measurement Perspectives Conceptual Framework for Library Economic Measures Inputs (Resources) Outputs (Products/ Services) Usage (Use & Nonuse) Outcomes (Consequences of Information) Domain LibraryUserIndustry/ Sector OrganizationSociety

14. Derived Measures Performance Effectiveness Specific Measures Measurement Perspectives Cost-Effectiveness Conceptual Framework for Library Economic Measures Inputs (Resources) Outputs (Products/ Services) Usage (Use & Nonuse) Outcomes (Consequences of Information) Domain LibraryUserIndustry/ Sector OrganizationSociety

15. Derived Measures Performance Effectiveness Specific Measures Measurement Perspectives Cost-EffectivenessImpact Conceptual Framework for Library Economic Measures Inputs (Resources) Outputs (Products/ Services) Usage (Use & Nonuse) Outcomes (Consequences of Information) Domain LibraryUserIndustry/ Sector OrganizationSociety

16. Derived Measures Performance Effectiveness Specific Measures Measurement Perspectives Cost-EffectivenessImpact Cost/Benefit; Return-on-Investment Conceptual Framework for Library Economic Measures Inputs (Resources) Outputs (Products/ Services) Usage (Use & Nonuse) Outcomes (Consequences of Information) Domain LibraryUserIndustry/ Sector OrganizationSociety

17. Evaluation Involves Comparison Over time Between services Between the current and the anticipated Between inputs and outputs/outcomes

18. ROI is a Comparison Compare Returns and Investments Investments or Costs -to the library (or service within) -to the library user -to the organization Returns -to the library -to the library user -to the organization

19. ROI is a Comparison Investments or Costs to the library (or service within): $ spent to the library user: time and $ spent to the organization: total time and $ spent

20. ROI is a Comparison Returns Important to distinguish between: outputs - produced as result of expenditure, e.g., larger collection use - extent to which outputs are used, e.g., increased use of library materials outcomes - consequences of use, e.g., learned something new, saved time, etc. Use contingent evaluation

21. Return-on-Investment Defined Investment –Library expenditures and organization overhead –User time in wages and overhead –Cost of other relevant resources Return –Contingent valuation of the additional cost to users if there were no library services –Changes in user outcomes e.g., productivity, information needs satisfied, and other relevant indicators

22. Definition of Contingent Valuation Contingent valuation is an economic method used to assess the benefits of non-priced goods and services (e.g., libraries or specific library services) by examining the implications of not having the product or service

23. Case Study 1:Library ROI (Availability & Use)

24. Investment - Total Library expenditures User expenditures Total organization investment

25. Special Library Investments Library expenditure –$610 per professional User cost to use library services –$1,090 per professional Total organization investment –$1,700 per professional

26. Contingent Valuation - Library Availability User cost to use library services User cost to use alternative source for service or information Net benefit or return (availability) is difference between current cost to use library and anticipated cost to use alternatives

27. Special Library ROI Net benefit or return: User additional cost to use alternatives $5,010 per professional Library ROI (availability) ($5,010÷$1,700) or 2.9 to 1

28. Returns from Library Use Amount of use of information resource or service Benefits derived from that use (time saved, $ saved, etc.) Assume a fixed willingness to invest When using costlier alternatives, some uses would be lost

29. Library ROI Benefits derived –$310 per journal article reading –$650 per book reading –$1,090 per internal report reading Total savings across all readings –$31,300 per professional for reading journal articles –$28,000 per professional for reading books –$42,500 for reading internal reports

30. Library ROI Labor ROI –Labor savings divided by time spent reading –8.3 to 1 for reading journals articles –7.9 to 1 for reading books –14.2 to 1 for reading internal reports. –Overall ROI (Labor) is 8.3 to 1 Lost Benefits $12,240 per professional Library ROI (Use) –Ranges from 5 to 1 …. 16 to 1

31. Outcomes of Special Library Use Special libraries help increase productivity –User time and/or other expenditures are saved in over one-third of library uses –User time and/or other expenditures are saved in about 40 percent of readings of library documents –Five indicators of user productivity are correlated with amount of library

32. Outcomes of Special Library Use Special libraries contribute to users quality of work Users indicate that that the library services are absolutely essential to their work for nearly 40 percent of uses Users indicate that the library services improves their quality of work in nearly 60 percent of uses Users whose work has been recognized through awards, etc. use libraries more than cohorts and non-award winners

33. Case Study 2: Library Journal Collection ROI

34. Trends in Special Library Journal Collection Use Examine patterns of information seeking Examples of scientists and engineers Amount of journal reading by scientists is up Reading from special library journal collections is up Increases due in part to electronic journals

35. Patterns of Information Seeking Focus on article reading Time spent reading How read articles are identified How the articles are obtained Format of the articles A trend in the influence of libraries Effect of e-journals on library cost

36. Many Ways to Identify Articles Browsing through print or electronic journals (mostly for current awareness) Searching in search engines in bibliographic and e-journal databases (mostly for research and writing) Follow-up of citations in journals and other publications Recommendations from colleagues, etc. Other (e.g., alerts, preprint services, etc.)

37. Many Sources of Articles Personal subscriptions Library collections Authors, colleagues, etc. Other

38. Where Do Readers Obtain Articles? Personal SubscriptionsDow n Library CollectionsUp From Another PersonSam e Author WebsitesSmal l Preprint and other databases Smal l Age of articles is important Trends

39. Amount of Reading Depends on Where Readers Works University scientists average reading more than non-university scientists served by special libraries –University scientists: 252 readings/year –Non-university scientists: 113 readings/year HOWEVER Non-University scientists account for 75 percent of all reading in U.S. AND University scientists write nearly 75 percent of articles published by U.S. authors

40. Trends in Reading Patterns of Scientists Served by Special Libraries The appear to be reading more They rely on libraries more Reasons for increased library use

41. Average Annual Reading by Scientists Served by Special Libraries

42. Average Annual Reading by Scientists Served by Special Libraries by Source of Articles Read

43. Reasons for Shift to Reading from Special Library Collections Decrease in personal subscriptions More reading of articles identified by online bibliographic searches Electronic collections have broadened access to articles

44. Average Number of Personal Subscriptions per Scientist

45. Average Number of Articles Identified by Online Search

46. Electronic Collections Contribution Personal subscriptions - 90% print Library collections –80% electronic –Broadens journal availability –Saves readers about 20 hours per year Breadth of reading has increased –Read from about 13 journals in 1977 –Over twice that amount now

47. Library Journal Collection Investment Library expenditure (amount allocated to faculty and staff use) –$1.87 million Faculty and staff cost to use the library collection –$1.56 million Total organization investment –$3.43 million

48. Library Journal Collection Return (Availability) User cost to use alternative sources of article information ($11.38 million in time, $2.1 million in purchases) $13.48 million Return/net benefit ($13.48 million - $3.43 million) $10.05 million

49. Library Journal Collection ROI (Availability) ROI (Availability) ($10.05 million ÷ $3.43 million) 2.9 to 1 PLUS Potential lost benefits in savings, productivity, etc.

50. Library Journal Collection Return (Use) If the journal collection were not available, professionals said they: Did not know37% Would use another library 28% Would use another source21% Would purchase the item 6% Would take another course of action 8%

51. Library Journal Collection Return (Use) About 25% of readings of library-provided journal articles saved the user time and/or money Average amount of savings $310 per article reading (special library studies) $385 per article reading (national studies of scientists and engineers) ROI (Use) 8.3 to 1

52. Library Journal Collection Return (Use) Reasons for savings Avoided having to do some work49% Provided confirmation of work27% Stopped unproductive line of work10% Modified research or design12% Modified analysis methods16%

53. Total Time Saved Total time saved –250,000 hours –114 FTEs Electronic remote access savings in time –50,000 hours –23 FTEs* * Based on an average of 2,200 hours worked

54. ROI Methods Library ROI Methods Journal Collection ROI Methods

55. Library ROI Methods Mailed survey to organization professionals –Letter from high level executive –Publicity by library –Promise results to users –Reminders by cards Universe is visits (critical incidents)

56. Library ROI Methods Observations include: Library use (including last use) –Amount of use –Services used –Time spent using library –Time and cost to use alternative sources if the library were not there

57. Library ROI Methods Observations include: Section on purposes and consequences of using the library –Purposes (e.g., research, writing, etc.) –Ways information/services affect purposes –Savings in time and/or money from information –Importance of information/services

58. Library ROI Methods Observations include: Section on demographics –Education level achieved, discipline –Awards received –Authorships

59. Methods: Cost of Users Time We use the average salary plus an amount for overhead From other studies, we estimate that professionals average about 2,200 hours of work per year. This allows us to calculate an hourly rate per hour for users.

60. Survey of Special Library Use Your responses are confidential and data will be reported only in aggregated form. Because your answers are extremely important to the accuracy of our study, please submit the questionnaire even if you are unable to answer all the questions. We have tried to keep the questionnaire as short and simple as possible and yet achieve our study objectives. If you have any questions, please contact….

61. Survey of Special Library Use Section 1: Library Use 1.In the past year (12 months) approximately how often have you used your (company, agency, laboratory) library? Uses include visiting the library in-person or remotely including access to the librarys bibliographic databases, electronic journal collections, to request services, etc. Number of times the library was used the past year: _______ times

62. Survey of Special Library Use The following questions in this section refer to the last use of the library, whether in-person or remotely. Note that this last use may not be typical, but will help us establish the range of uses of your library. 2. Was this last use in-person or remote? – In-person – Remote By e-mail By telephone 3. How long ago was this last use? 4. What services were used during this last use?

63. Survey of Special Library Use 5. About how much time did you spend on this last use of the library? 6. If you did not have the library, what would you have done to obtain the information or service obtained during this last use? –I would not bother getting the information (skip to question 7) – I need the information, but do not know where else to get it (skip to question 7) – I would obtain the information from another source. Please specify source here: _____________________

64. Journal Collection ROI Methods Survey of professionals In-depth cost of library collection purchase and processing

65. Reader Survey Methods Mailed survey… Universe is readings (critical incident) See questionnaire sample in handout

66. Journal Collection Costs Include All Resources Used Collection purchases Staff Facilities Equipment and systems Photocopies, binding, etc.

67. Allocating Library Staff Costs Prepare a list of relevant journal processing activities Prepare a staff activity log Instruct library staff for filling out the logs Establish detailed calculation methods

68. Total Cost and Cost Per Use of Collection Access Services

69. Life-Cycle Cost per Title Electronic collection - $180 per title Print collection - $580 per title –Current collection - $190 per title –Backfile collection - $390 per title

70. Annual Cost Per Reading Compare the Unit Cost of Services –Electronic - $3.00 per reading –Current Periodicals - $13.60 per reading –Bound Backfiles - $15.60 per reading –ILL - $8.40 per item –ILB - $12.60 per item Life-Cycle Cost Per Reading Electronic - $7.30 per title Print - $23.50 per title

71. Communicating Results

72. Target Audiences Identify your target audiences – who do you want to pay attention to your information?

73. Target Audience Characteristics Summarize key characteristics of your target audience(s) Current key issues and concerns Language/educational level Preferred communication media

74. Need Sound Bytes Core messages So therefore… Meaning and impact Comprehensibility

75. Core Messages List of 1-sentence statements of main points, NOT long paragraphs Maximum of 4 to 5 statements Sequenced from most important to least Need to have a 1-sentence summary for all

76. So Therefore… Your core messages must include not only facts and figures but also what it means This is true, so therefore –You should… –We will… –This proves… –Etc.

77. Core Messages: Meaning and Impact Your core messages must answer the question for your target audience: What does it mean and why should I care??

78. Core Messages: Comprehensibility Simplify, simplify, simplify! Just because you know big words doesnt mean you have to use them

79. Core Messages: Comprehensibility Insofar as the comprehensive research is too voluminous to delineate here, if you and/or your corporation, organization, etc. are apprehensive as to the pedagological or methodological veracity, authenticity or rationale supporting the aforesaid investigation please reference the relevant available electronic resource. Compared to: Please check our website for more information.

80. Core Messages: Comprehensibility Avoid being misquoted in the press (including the academic press!) by asking the reporter to: E-mail you his/her questions before the interview. Repeat back to you what you have said Call you back with any additional questions, as necessary. Provide you with a copy of your quotes before the article is published. Quickly correct any inaccuracies that you find. (Some reporters wil not provide you with pre-publication quotes, but some will. And it won't hurt to ask.)

81. Craft Your Message Identify the desired results of your communication Target AudienceKnowFeelDo Organizational leadership The results of the study: the numbers and their implications The library/in- formation service is well worth the cost Support additiona l funding requests

82. Craft Your Message Attention and sequencing - pyramid approach Find or create visuals Humanize your message Try out your message with your harshest critic and dont argue with their comments - listen and learn

83. Attention and Sequencing Must have an opening line (headline, first line of interview,etc.) that is attention getting focuses on impacts rather than data Need excellent headline/title Visuals

84. Visuals Communicate Without Special LibraryWith Special Library Time of One Information Professional Saves the equivalent of FIVE professionals time

85. Follow Up Every communication is the foundation for the next Be clear about asking for what you want Be appreciative

86. José-Marie Griffiths, PhD Sarah E. Aerni Email: jmgriff@unc.edujmgriff@unc.edu Email: aerni@email.unc.edu School of Information and Library Science University of North Carolina at Chapel Hill phone: (919) 962-8363

87. Thanks for coming! Please submit the evaluation forms to us so that we can send them back to SLA. Enjoy the rest of the conference!