©2014 MFMER | slide-1 Unlocking a Quality Management System’s Potential – Key Components and Self- Monitoring Tools

©2014 MFMER | slide-2 Agenda Introduction to the Research shield at Mayo Clinic Circumstances that led us to a Quality Management System (QMS) Key components of a QMS Self-monitoring tools Control limits and defined escalation

©2014 MFMER | slide-3 Objectives Explain key components of an effective QMS Illustrate helpful tools that can be used to self-monitor Discuss importance of control limits and defined escalation processes

©2014 MFMER | slide-4 Mayo Clinic Research Shield Physicians actively engaged in research ~ 640 Individuals certifying research effort > 4000 Research space > 900,000 sq. ft. Grants and contracts > 4700 In the top 25 NIH grant funding by institutions Active Institutional Review Board-approved human research studies > 8,900 Research and review articles in peer-reviewed journals > 5,700

©2014 MFMER | slide-5 Research Funding Sources 61% 11%

©2014 MFMER | slide-6 Research Finance at Mayo Clinic Research Finance is an enterprise wide division with staff working in Arizona, Florida, and Rochester. The team is devoted to ensuring compliant financial processes for post award activity for all funding types. Organizationally in the Finance Department Research Finance has employees working across 7 functional teams Award Management Award Compliance Award Revenue Cores Program Services Reporting Quality

©2014 MFMER | slide-7 Mayo Clinic Research Finance Historical Staffing Staffing numbers Pre Department of Justice (DOJ) investigation – 2003 – 25 FTE Post implementation of controls and launch of QMS – 2008 – 100 FTE QMS additions and enhancement, Self- monitoring in place, Working Differently initiative – 2013 – 88 FTE

©2014 MFMER | slide-8 DOJ Investigation and Settlement Whistleblower (qui tam) suit filed – Agents visit DOJ press release and investigation closed Mayo settled for $6.5M Reviewed NIH funds spent from November 2003 May 2005 June 2005 November May 2005

©2014 MFMER | slide-9 Transition In June 2005, NIH sent a letter of inquiry asking for compliance related information and assurances related to 4 areas: Cost Transfers Financial Management Systems Written Policies and Procedures Roles, Responsibilities, and Training

©2014 MFMER | slide-10 Mayo Clinic Response ( ) Building the Framework Conducted comprehensive gap analysis Identified opportunities for improved compliance controls Self-disclosed to NIH self-identified compliance issues Met with NIH officials in Bethesda, multiple times to provide comprehensive updates

©2014 MFMER | slide-11 Mayo Clinic Response ( ) Building the Framework Executed 26 projects Automation Policies & Procedures Roles & Responsibilities Tools Implemented new Enterprise Resource Planning (ERP) system, including Grants Management module

©2014 MFMER | slide-12 Strengthening the Framework ( ) In 2008, adopted a Quality Management System, modeled after International Organization of Standardization (ISO) framework Cornerstones remain; Added continuous improvement, competencies, and customer satisfaction elements Enhanced training, measurement and analysis and our documentation system

©2014 MFMER | slide-13 Quality Management System - Key Components Continuous Improvement Customer Service & Satisfaction Culture QMS Administration Documentation System Management Commitment Measurement & Analysis Competence, Awareness & Training Start

©2014 MFMER | slide-14 Research Finance Mission Statement Mayo Clinic Research Finance believes in quality management. Research Finance aspires to use quality as the vehicle behind all it does in an effort to meet and exceed our customers’ expectations. The Research Finance Quality Management System emphasizes a proactive approach to meeting our customers’ needs and to ensure continuous improvement. Management Commitment

©2014 MFMER | slide-15 Establishing a Quality Culture Work environment Readiness Employee engagement and empowerment Expectations Accountability Promote teamwork Promote continuous improvement Culture

©2014 MFMER | slide-16 Self-Monitoring Tools

©2014 MFMER | slide-17 QMS Annual Plan Critical part of a Quality Management System Identifies long term goals and current year objectives Commits required resources in writing Provides staff with direction for coming year Management Commitment Culture QMS Administration

©2014 MFMER | slide-18 Annual Plan Content Management Message: Communicates expectations, incorporates QMS elements and conveys that the QMS plan is owned by the department Define Scope: Identifies area that QMS supports Purpose of the Department’s Quality Plan Describes actions taken to ensure core business processes and QMS are continually improved

©2014 MFMER | slide-19 Annual Plan Content Goals: Communicates long term purpose for plan, doesn’t need to be measureable Objectives: Define time frame, objectives are specific and measureable Activities by QMS Element Align QMS work identified within the plan with the eight elements Quality Plan Administration Identifies responsibilities of the plan administrator

©2014 MFMER | slide-20 Policies, Procedures and Work Instructions Policy – organization’s official position setting an expectation of behavior Procedure – high level of roles and responsibilities explaining how it happens Work Instruction – documents how to perform the work Record Retention Documentation System

©2014 MFMER | slide-21 Stores policies, procedures and work instructions Automated s or monthly reports Identifies person/area responsible for documents Check in/out, version control, and historical content available Content Management System

©2014 MFMER | slide-22 Competence, Awareness & Training Employee performance appraisals Learning management system Training Education based format (utilize Bloom’s Taxonomy) Competency assessment Competence, Awareness & Training

©2014 MFMER | slide-23 Bloom’s Taxonomy

©2014 MFMER | slide-24 Standard Training Format Title Objectives utilizing Bloom’s Taxonomy Define Cost Sharing Introduce and discuss each objective Test your knowledge questions “____________or matching means that portion of project or program costs not borne by the Federal Government.” Summary restating objectives Cost sharing is the portion of project costs not borne by the Federal Government.

©2014 MFMER | slide-25 Bloom’s Taxonomy

©2014 MFMER | slide-26 Standard Training Format Title Objectives utilizing Bloom’s Taxonomy Define Cost Sharing Introduce and discuss each objective Test your knowledge questions “____________or matching means that portion of project or program costs not borne by the Federal Government.” Summary restating objectives Cost sharing is the portion of project costs not borne by the Federal Government.

©2014 MFMER | slide-27 Competency Plan 1. Conduct needs assessment 2. Identify competencies addressed by job 3. Write goals and objectives 4. Determine teaching methods 5. Determine assessment methods 6. Determine program improvement methods

©2014 MFMER | slide-28 Measurement and Analysis Balanced Scorecard Outcomes Financial Processes People Strategic Plan Measurement & Analysis

©2014 MFMER | slide-29 Measurement and Analysis Compliance - Internal Control Evaluation (C-ICE) Scoping Flows & Controls Testing C-ICE Metrics

©2014 MFMER | slide-30 Customer Service & Satisfaction What is customer service? Customer service is exceeding the needs of both internal and external customers. Tools: Event Management Voice of the Customer (VOC) Customer Service & Satisfaction

©2014 MFMER | slide-31 Continuous Improvement 1. Establish Scope 2. Identify & Define Reportable Categories 3. Report Results 4. Outline Escalation Quality Assurance Continuous Improvement

©2014 MFMER | slide-32 Quality Assurance Controls 90 % of all QA findings are process issues

©2014 MFMER | slide-33 Continuous Improvement 1. Establish Scope 2. Identify & Define Reportable Categories 3. Report Results 4. Outline Escalation Quality Assurance

©2014 MFMER | slide-34 Quality Assurance Control Chart

©2014 MFMER | slide-35 Quality Assurance Tools

©2014 MFMER | slide-36 Project Management vs. DMAIC

©2014 MFMER | slide-37 Project Management (PMBOK)

©2014 MFMER | slide-38 Linking DMAIC and Change * * Many change management tools can (and should) be used in more than one phase.

©2014 MFMER | slide-39 Control Limits and Defined Escalation Processes

©2014 MFMER | slide-40 Self-Monitoring QMS Identify controls Visual Easily understood Defined escalation Communicated Administered

©2014 MFMER | slide-41 Policies, procedures and work instructions warehoused in content management system Expectations and due dates communicated Documentation Controls

©2014 MFMER | slide-42 Balanced Scorecard

©2014 MFMER | slide-43 Documentation Controls

©2014 MFMER | slide-44 QMS Metrics Timely Federal Financial Report (FFR) submission Past due employee performance appraisals Completion of assigned compliance training Successfully passing the Competency Assessment Completion of C-ICE flows, controls and testing Timely filing of documents for Record Retention

©2014 MFMER | slide-45 Quality Assurance Control Chart

©2014 MFMER | slide-46 Quality Assurance Controls 90 % of all QA findings are process issues

©2014 MFMER | slide-47 Does it really work? Continuous Improvement Customer Service & Satisfaction Culture QMS Administration Documentation System Management Commitment Measurement & Analysis Competence, Awareness & Training

©2014 MFMER | slide-48 QMS – Before and After Vision Mission Goals / Objectives Vision Mission Goals / Objectives Before QMS QMS Implementation After QMS

©2014 MFMER | slide-49 Indicators of Success Staff Costs: FTE reduced nearly 15% over the past two years as a result of process improvement Process Improvements: 35 process improvement projects were completed in the last 9 months of 2013 Accomplishments

©2014 MFMER | slide-50 Mayo Clinic A-133 Audit Trend (Research Specific)

©2014 MFMER | slide-51 Summary Implement a Quality Management System A QMS consists of eight key components Self-monitoring tools help maintain an efficient and effective QMS Control limits and defined escalation processes need to be understood

©2014 MFMER | slide-52 Questions/Discussion