1. ©2014 MFMER | slide-1 Unlocking a Quality Management System’s Potential – Key Components and Self- Monitoring Tools

2. ©2014 MFMER | slide-2 Agenda Introduction to the Research shield at Mayo Clinic Circumstances that led us to a Quality Management System (QMS) Key components of a QMS Self-monitoring tools Control limits and defined escalation

3. ©2014 MFMER | slide-3 Objectives Explain key components of an effective QMS Illustrate helpful tools that can be used to self-monitor Discuss importance of control limits and defined escalation processes

4. ©2014 MFMER | slide-4 Mayo Clinic Research Shield Physicians actively engaged in research ~ 640 Individuals certifying research effort > 4000 Research space > 900,000 sq. ft. Grants and contracts > 4700 In the top 25 NIH grant funding by institutions Active Institutional Review Board-approved human research studies > 8,900 Research and review articles in peer-reviewed journals > 5,700

5. ©2014 MFMER | slide-5 Research Funding Sources 61% 11%

6. ©2014 MFMER | slide-6 Research Finance at Mayo Clinic Research Finance is an enterprise wide division with staff working in Arizona, Florida, and Rochester. The team is devoted to ensuring compliant financial processes for post award activity for all funding types. Organizationally in the Finance Department Research Finance has employees working across 7 functional teams Award Management Award Compliance Award Revenue Cores Program Services Reporting Quality

7. ©2014 MFMER | slide-7 Mayo Clinic Research Finance Historical Staffing Staffing numbers Pre Department of Justice (DOJ) investigation – 2003 – 25 FTE Post implementation of controls and launch of QMS – 2008 – 100 FTE QMS additions and enhancement, Self- monitoring in place, Working Differently initiative – 2013 – 88 FTE

8. ©2014 MFMER | slide-8 DOJ Investigation and Settlement Whistleblower (qui tam) suit filed – Agents visit DOJ press release and investigation closed Mayo settled for $6.5M Reviewed NIH funds spent from 1992-2003 November 2003 May 2005 June 2005 November 2003 - May 2005

9. ©2014 MFMER | slide-9 Transition In June 2005, NIH sent a letter of inquiry asking for compliance related information and assurances related to 4 areas: Cost Transfers Financial Management Systems Written Policies and Procedures Roles, Responsibilities, and Training

10. ©2014 MFMER | slide-10 Mayo Clinic Response (2005-2008) Building the Framework Conducted comprehensive gap analysis Identified opportunities for improved compliance controls Self-disclosed to NIH self-identified compliance issues Met with NIH officials in Bethesda, multiple times to provide comprehensive updates

11. ©2014 MFMER | slide-11 Mayo Clinic Response (2005-2008) Building the Framework Executed 26 projects Automation Policies & Procedures Roles & Responsibilities Tools Implemented new Enterprise Resource Planning (ERP) system, including Grants Management module

12. ©2014 MFMER | slide-12 Strengthening the Framework (2008-2013) In 2008, adopted a Quality Management System, modeled after International Organization of Standardization (ISO) framework Cornerstones remain; Added continuous improvement, competencies, and customer satisfaction elements Enhanced training, measurement and analysis and our documentation system

13. ©2014 MFMER | slide-13 Quality Management System - Key Components Continuous Improvement Customer Service & Satisfaction Culture QMS Administration Documentation System Management Commitment Measurement & Analysis Competence, Awareness & Training Start

14. ©2014 MFMER | slide-14 Research Finance Mission Statement Mayo Clinic Research Finance believes in quality management. Research Finance aspires to use quality as the vehicle behind all it does in an effort to meet and exceed our customers’ expectations. The Research Finance Quality Management System emphasizes a proactive approach to meeting our customers’ needs and to ensure continuous improvement. Management Commitment

15. ©2014 MFMER | slide-15 Establishing a Quality Culture Work environment Readiness Employee engagement and empowerment Expectations Accountability Promote teamwork Promote continuous improvement Culture

16. ©2014 MFMER | slide-16 Self-Monitoring Tools

17. ©2014 MFMER | slide-17 QMS Annual Plan Critical part of a Quality Management System Identifies long term goals and current year objectives Commits required resources in writing Provides staff with direction for coming year Management Commitment Culture QMS Administration

18. ©2014 MFMER | slide-18 Annual Plan Content Management Message: Communicates expectations, incorporates QMS elements and conveys that the QMS plan is owned by the department Define Scope: Identifies area that QMS supports Purpose of the Department’s Quality Plan Describes actions taken to ensure core business processes and QMS are continually improved

19. ©2014 MFMER | slide-19 Annual Plan Content Goals: Communicates long term purpose for plan, doesn’t need to be measureable Objectives: Define time frame, objectives are specific and measureable Activities by QMS Element Align QMS work identified within the plan with the eight elements Quality Plan Administration Identifies responsibilities of the plan administrator

20. ©2014 MFMER | slide-20 Policies, Procedures and Work Instructions Policy – organization’s official position setting an expectation of behavior Procedure – high level of roles and responsibilities explaining how it happens Work Instruction – documents how to perform the work Record Retention Documentation System

21. ©2014 MFMER | slide-21 Stores policies, procedures and work instructions Automated emails or monthly reports Identifies person/area responsible for documents Check in/out, version control, and historical content available Content Management System

22. ©2014 MFMER | slide-22 Competence, Awareness & Training Employee performance appraisals Learning management system Training Education based format (utilize Bloom’s Taxonomy) Competency assessment Competence, Awareness & Training

23. ©2014 MFMER | slide-23 Bloom’s Taxonomy http://nashvilleisd.groupsite.com/uploads/files/x/000/04b/a15/performance_verbs_for_learning_objectives.png

24. ©2014 MFMER | slide-24 Standard Training Format Title Objectives utilizing Bloom’s Taxonomy Define Cost Sharing Introduce and discuss each objective Test your knowledge questions “____________or matching means that portion of project or program costs not borne by the Federal Government.” Summary restating objectives Cost sharing is the portion of project costs not borne by the Federal Government.

25. ©2014 MFMER | slide-25 Bloom’s Taxonomy

26. ©2014 MFMER | slide-26 Standard Training Format Title Objectives utilizing Bloom’s Taxonomy Define Cost Sharing Introduce and discuss each objective Test your knowledge questions “____________or matching means that portion of project or program costs not borne by the Federal Government.” Summary restating objectives Cost sharing is the portion of project costs not borne by the Federal Government.

27. ©2014 MFMER | slide-27 Competency Plan 1. Conduct needs assessment 2. Identify competencies addressed by job 3. Write goals and objectives 4. Determine teaching methods 5. Determine assessment methods 6. Determine program improvement methods

28. ©2014 MFMER | slide-28 Measurement and Analysis Balanced Scorecard Outcomes Financial Processes People Strategic Plan Measurement & Analysis

29. ©2014 MFMER | slide-29 Measurement and Analysis Compliance - Internal Control Evaluation (C-ICE) Scoping Flows & Controls Testing C-ICE Metrics

30. ©2014 MFMER | slide-30 Customer Service & Satisfaction What is customer service? Customer service is exceeding the needs of both internal and external customers. Tools: Event Management Voice of the Customer (VOC) Customer Service & Satisfaction

31. ©2014 MFMER | slide-31 Continuous Improvement 1. Establish Scope 2. Identify & Define Reportable Categories 3. Report Results 4. Outline Escalation Quality Assurance Continuous Improvement

32. ©2014 MFMER | slide-32 Quality Assurance Controls 90 % of all QA findings are process issues

33. ©2014 MFMER | slide-33 Continuous Improvement 1. Establish Scope 2. Identify & Define Reportable Categories 3. Report Results 4. Outline Escalation Quality Assurance

34. ©2014 MFMER | slide-34 Quality Assurance Control Chart

35. ©2014 MFMER | slide-35 Quality Assurance Tools

36. ©2014 MFMER | slide-36 Project Management vs. DMAIC

37. ©2014 MFMER | slide-37 Project Management (PMBOK)

38. ©2014 MFMER | slide-38 Linking DMAIC and Change * * Many change management tools can (and should) be used in more than one phase.

39. ©2014 MFMER | slide-39 Control Limits and Defined Escalation Processes

40. ©2014 MFMER | slide-40 Self-Monitoring QMS Identify controls Visual Easily understood Defined escalation Communicated Administered

41. ©2014 MFMER | slide-41 Policies, procedures and work instructions warehoused in content management system Expectations and due dates communicated Documentation Controls

42. ©2014 MFMER | slide-42 Balanced Scorecard

43. ©2014 MFMER | slide-43 Documentation Controls

44. ©2014 MFMER | slide-44 QMS Metrics Timely Federal Financial Report (FFR) submission Past due employee performance appraisals Completion of assigned compliance training Successfully passing the Competency Assessment Completion of C-ICE flows, controls and testing Timely filing of documents for Record Retention

45. ©2014 MFMER | slide-45 Quality Assurance Control Chart

46. ©2014 MFMER | slide-46 Quality Assurance Controls 90 % of all QA findings are process issues

47. ©2014 MFMER | slide-47 Does it really work? Continuous Improvement Customer Service & Satisfaction Culture QMS Administration Documentation System Management Commitment Measurement & Analysis Competence, Awareness & Training

48. ©2014 MFMER | slide-48 QMS – Before and After Vision Mission Goals / Objectives Vision Mission Goals / Objectives Before QMS QMS Implementation After QMS

49. ©2014 MFMER | slide-49 Indicators of Success Staff Costs: FTE reduced nearly 15% over the past two years as a result of process improvement Process Improvements: 35 process improvement projects were completed in the last 9 months of 2013 Accomplishments

50. ©2014 MFMER | slide-50 Mayo Clinic A-133 Audit Trend (Research Specific)

51. ©2014 MFMER | slide-51 Summary Implement a Quality Management System A QMS consists of eight key components Self-monitoring tools help maintain an efficient and effective QMS Control limits and defined escalation processes need to be understood

52. ©2014 MFMER | slide-52 Questions/Discussion