1 Performance Management Presentation Provide Basic Animal Life Support Team Members: Eileen Morgan, Dr. Charmaine Foltz, Jim Weed, Dennis Barnard, Andy Limerick, Alphie Cisar, Brent Morse, Wayne O'Brien, Sonia Love ORS National Institutes of Health 14 January, 2005

2 Table of Contents Main Presentation Performance Management Plan Customer Perspective C1a: AAALAC accreditation ratings C1b:ACUC inspection ratings by type C2c: Percentage of non-exempt animal census requiring socialization C2d:Percentage of attempted socializations that were successful C3a: ORS Customer Scorecard ratings for animal husbandry and clinical veterinary and technical services Internal Business Process Perspective IB1a: Investigator response time to notifications of health status IB1b: Percentage of holding agreements processed by beginning of fiscal year

3 Table of Contents (cont.) Main Presentation Learning and Growth Perspective LG1a: Percentage of staff with desired credentials LG2a: Percentage of technicians fully trained on all procedures by facility LG2b: Percentage of facilities with one fully trained technician Financial Perspective F1a: Unit cost of animal husbandry services F1b:Unit cost of clinical veterinary and technical services F1c:Unit cost of special environmental factors services F2a: Overall percentage of available housing utilized F2b: Overall percentage increase in holding capacity F3a: Percentage dollar savings associated with caging purchase strategy change Conclusions Appendix

4 Performance Management Plan

5 Relationship Among Performance Objectives --- relationship between objectives in different perspectives ___ relationship between objectives within a perspective most important objectives

6 Customer Perspective

7

8 C1a: AAALAC Accreditation Ratings FY02 fully accredited with commendations for DVR FY99 fully accredited FY93 all of NIH fully accredited; DVR falls under that accreditation DVR continuously accredited since 1960s Note: AAALAC site visits are tri-annual 1960sFY02FY99 FY93

9 C1b: ACUC Inspection Results by Type Note: Minor requests are not necessarily deficiencies, but rather suggestions.

10 Customer Perspective (cont.)

11 C2c: Percentage of Non-Exempt Animal Census Requiring Socialization FY04 56% of the non-exempt animal census were socialized FY03 50% of the non-exempt animal census were socialized FY02 46% of the non-exempt animal census were socialized Number of Animals

12 C2d: Percentage of Attempted Socializations That Were Successful FY04 81% of the attempted socializations were successful FY03 84% of the attempted socializations were successful FY02 90% of the attempted socializations were successful Number of Animals

13 Customer Perspective (cont.)

14 Customer Scorecard Methodology In FY04, modified ORS Customer Scorecard for animal husbandry and clinical veterinary and technical services

15 C3a: Customer Scorecard Ratings for Animal Husbandry and Clinical Veterinary and Technical Services Survey was distributed via to APDs and PIs. Data Collection began May 28, Reminder s sent June 11, 2004 and July 1, Data collection ended July 20, % response rate (N = 18 out of 92) 39% (N = 7) were APDs and 61% (N = 11) were PIs

16 C3a: Overall Satisfaction ratings with Animal Husbandry Services during the past 12 months Unsatisfactory Outstanding N = 18

17 C3a: Overall Satisfaction ratings with Clinical Veterinary and Technical Services during the past 12 months Unsatisfactory Outstanding N = 14

18 Customer Perspective Summary ACUC ratings included 46 commendations, 77 minor requests, and 15 major deficiencies Attempted socializations of non-human primates increased from 50% to 56% in FY04 81% of the attempted socializations were successful Customer survey was completed in July, FY04. The overall mean on customer satisfaction for animal husbandry is 7.4 and for clinical veterinary and technical is 7.7. These means are about 2 points above FY02 ratings for each of the services.

19 Customer Perspective Planned Actions ACUC semi-annual reviews of program and facilities ongoing; Strive to minimize major deficiencies Increase percentages of attempted and successful socializations for non-human primates Customer survey is currently planned to be repeated FY06 to maximize response

20 Internal Business Process Perspective

21 Internal Business Process Perspective

22 IB1a: Investigator Response Time to Notifications of Health Status Note: Data gathered from building 14C

23 IB1b: Percentage of Holding Agreements Processed by Beginning of Fiscal Year (as of 30 Sept) Note: Zero percentage of holding agreements were processed by beginning of fiscal year in FY03

24 Internal Business Process Perspective Summary Of Investigators who responded, 86% responded within 24 hours 56% of Investigators responded Holding agreements processed by first day in fiscal year continues to be 100%

25 Internal Business Process Perspective Planned Actions Response time by PI’s will continue to be monitored to ensure appropriate veterinary care Continue 100% success at processing holding agreements by beginning of FY

26 Learning and Growth Perspective

27 Learning and Growth Perspective

28 LG1a: Percentage of Staff with Desired Credentials

29 LG2a: Percentage of Technicians Fully Trained on all Procedures by Facility (Small Animals) 80% of technicians are fully trained on all procedures in sample facilities Technician’s in sample facilities are trained on 95% of proficiencies Note: Facilities include 10A, 127/128, 28ICU, 28 D wing, 14 F/G

30 LG2b: Percentage of Facilities with One Fully Trained Technician (Small Animals) 100% of facilities have one trained technician in all competencies listed. Note: Facilities include 10A, 127/128, 28ICU, 28 D wing, 14 F/G

31 Learning and Growth Perspective Summary Staff credentials Recruiting and retaining veterinarians with ACLAM board certification challenging Facility managers maintained AALAS LATG certification status Technical proficiency Data was obtained on 56% of small animal facilities where 80% of technicians are fully trained on all procedures and 100% of the facilities have one fully trained technician.

32 Learning and Growth Perspective Planned Actions Credentials Strive to obtain and retain board certified veterinarians in all new hires Encourage facility management personnel to obtain LATG certification Technical training proficiency Data collected on small animal facilities will be expanded to include all small animal facilities Tracking will begin in FY05 on large animal proficiencies

33 Financial Perspective

34 Financial Perspective

35 F1a: Animal Husbandry Unit Cost

36 F1b: Clinical Veterinary and Technical Services Unit Cost

37 F1c: Special Environmental Factors Unit Cost

38 Financial Perspective (cont)

39 F2a: Overall Percentage Increase in Available Housing Utilized Note: Includes all housing available for all animals. Individual building and animal percentages are presented in separate graphs in the Appendix. 5.34% 2.73%

40 F2b: Overall Percentage Increase in Holding Capacity Note: Includes all housing available for all animals. Individual building and animal percentages are presented in separate graphs in the Appendix. 3.20% 7.91%

41 Financial Perspective (cont)

42 F3a: Percentage Dollar Savings in Caging and Equipment Purchases In FY02, identified bulk purchasing as potential improvement to streamline purchasing process

43 Financial Perspective Summary Unit Cost Animal Husbandry Unit Cost is $1.82/animal cage day Veterinary and Technical Unit Cost is $11.25/animal census Special Environmental Factors combined are $14.36/unit Housing Utilized and Holding Capacity continued to increase across DVR Bulk purchasing of caging resulted in 22% percent savings to DRV in FY04

44 Financial Perspective Planned Actions Unit Costs will continue to be calculated in future FY’s for cost containment purposes Housing Utilized Maintain or increase leasing agreements Housing Capacities continue to grow Use of new caging schemes, i.e., ventilated rodent racks Assess need to renovate areas in light of budgetary constraints

45 Financial Perspective Planned Actions Savings in Caging Purchases Continue to purchase bulk caging and equipment to obtain best pricing

46 Conclusions

47 Conclusions from PMP Full AAALAC Accreditation was maintained Continued monitoring ACUC semi annual reviews Socializations were increased over FY02 and FY03 Investigator’s who respond, do so within 24 hours Maintained Leasing Agreement Process consistent with FY03 – 100% completed by first day of new FY Identified an upward trend in Veterinary Certifications

48 Conclusions from PMP Tracking of Technician Proficiencies 56% of small animal facilities were evaluated 80% of technicians are fully trained on all proficiencies in sample facilities Of the 5 facilities evaluated, each identified one fully trained technician Housing Utilized and Capacity Continued to be innovative in use of animal housing space utilizing new technology and transforming space to meet demand for housing different species

49 Conclusions from PMP Highlight initiatives for FY05 To successfully complete FY05 AAALAC program review and site visit. Strive to continue to increase socializations Review response time data and process to ensure appropriate veterinary care Complete technician proficiencies for each veterinary technician; implement large animal proficiency tracking; strive for increased percentage of fully trained technicians (Use Initiative Roadmap here) – should be the same.

50 Appendix Pages 2 – 9 of Performance Management Plan Customer Perspective C1c: ACUC inspection ratings by facility C1d: ACUC inspection ratings by discrete service C2a:For control special environmental factors, number of consultations, presentations and trainings C2b: For control special environmental factors, number of publications and abstracts C3b: ORS Customer Scorecard Ratings for overall DVR services C3c: Number of commendations, minor requests/complaints, and major defects by type C3d: Number of commendations, minor requests/complaints, and major defects by facility C3e:Number of commendations, minor requests/complaints, and major defects by discrete service

51 Appendix (cont.) Appendix Internal Business Process Perspective IB1c: Average number of days to receive quality assurance test results on feed IB1c1: Control chart analysis of feed quality test assurance results turnaround time by month Learning and Growth Perspective

52 Appendix (cont.) Appendix Financial Perspective F2c:Percentage increase in holding capacity and housing utilized for building 10A rodents F2d: Percentage increase in holding capacity and housing utilized for building 14B rodents F2e: Percentage increase in holding capacity and housing utilized for building 14C rodents F2f: Percentage increase in housing utilized for building 14B non- human primates F2g: Percentage increase in housing utilized for buildings 103 and 104 non-human primates

53 Performance Management Plan (page 2)

54 Performance Management Plan (page 3)

55 Performance Management Plan (page 4)

56 Performance Management Plan (page 5)

57 Performance Management Plan (page 6)

58 Performance Management Plan (page 7)

59 Performance Management Plan (page 8)

60 Performance Management Plan (page 9)

61 Customer Perspective

62 C1c: ACUC Inspection Ratings by Facility Pilot tested log in FY04

63 Customer Perspective (cont.)

64 C2a: Number of Consultations, Presentations and Trainings Note: Includes Control Special Environmental Factors data

65 C2b: Number of Publications and Abstracts/Posters Note: Includes Control Special Environmental Factors data

66 Customer Perspective (cont.)

67 C3b: Customer Scorecard Ratings for Overall DVR Services NIH level animal services study to be completed FY05

68 C3c: Customer Feedback Categorization by Type Developed customer log of commendations and complaints Pilot results for FY04 Commendations 98 Complaints 33

69 C3d: Customer Feedback Categorization by Facility Developed customer log of commendations and complaints by facility

70 C3e: Customer Feedback Categorization by Discrete Service Developed customer log of commendations and complaints Will pilot test and revise in FY05

71 Internal Business Process Perspective

72 Internal Business Process Perspective

73 IB1c: Average Number of Days to Receive Quality Assurance Test Results on Feed Note: Control chart 1B1c1 shown in Appendix beginning with FY03 data Average =17 days Range = 6 – 27 days

74 IB1C1: Average Turnaround Time for Feed Quality Assurance Results

75 Financial Perspective

76 F2c: Percentage Increase in Holding Capacity and Housing Utilized for Building 10A Rodents Note: Percentage increase shown is against FY99 baseline

77 F2d: Percentage Increase in Holding Capacity and Housing Utilized for Building 14B Rodents Note: Percentage increase shown is against FY01 baseline

78 F2e: Percentage Increase in Holding Capacity and Housing Utilized for Building 14C Rodents Note: Percentage increase/decrease shown is against FY00 baseline FY02FY03FY04FY01

79 F2f: Percentage Increase in Housing Utilized for Building 14B Non-Human Primates Note: Percentage increase shown is against FY01 baseline

80 F2g: Percentage Increase in Housing Utilized for Buildings 103 and 104 Non- Human Primates Note: Percentage increase/decrease shown is against FY00 baseline FY01FY02FY03FY04