1. Workgroup Discussion Implementation, Certification, and Testing Workgroup Elizabeth Johnson, Co-Chair Christopher Ross, Co-Chair January 15, 2015

2. Health IT Congressional Provision December 16, 2015 continuing resolution omnibus spending bill directs ONC to, “take steps to decertify products that proactively block the sharing of information because those practices frustrate congressional intent, devalue taxpayer investments in (certified EHR technology) and make CEHRT less valuable and more burdensome for eligible hospitals and eligible providers to use.”spending bill ONC Requirements – ONC plan due to congress [by March 2015] Detailing the data sharing including estimate of the number of vendors or eligible hospitals and professionals who block information and a plan for addressing the issue – Health IT Policy Committee [will] submit a report by [December 2015] …on challenges and barriers to interoperability including: the technical, operational and financial barriers to interoperability, the role of certification in advancing or hindering interoperability across various providers, as well as any other barriers identified by the Policy Committee 11 Source: http://www.ihealthbeat.org/articles/2014/12/17/obama-signs-federal-spending-package-with-health-it-implicationshttp://www.ihealthbeat.org/articles/2014/12/17/obama-signs-federal-spending-package-with-health-it-implications *http://docs.house.gov/billsthisweek/20141208/CPRT-113-HPRT-RU00-HR83sa.pdf

3. Other Health IT Provisions within the Bill AgencyCharge Veterans Affairs (VA) Report to Congress on progress in updating EHR systems Various IT, including funding for the development of a standard data reference terminology model Flexibility to designate funds allocated for "Medical Services" and "Medical Support and Compliance" to enhance its VistA EHR system and interoperability Prohibits VA from spending more than 25% of allocated funds until it reports on its plan to achieve interoperability with DOD's health systems Department of Defense (DoD) Report to Congress on progress in updating EHR systems Like VA, will only will be able to spend 25% of the funding allocated to its Defense Health Program until it reports on its plan to achieve interoperability with VA's health system, including the implementation timeline and cost estimate of its new EHR system Health Resources and Services Administration (HRSA) Help small, rural hospitals adopt health IT Fund telehealth Department of Agriculture Fund broadband transmission in rural areas for telehealth and distance learning programs 2 Source: http://www.ihealthbeat.org/articles/2014/12/17/obama-signs-federal-spending-package-with-health-it-implicationshttp://www.ihealthbeat.org/articles/2014/12/17/obama-signs-federal-spending-package-with-health-it-implications *http://docs.house.gov/billsthisweek/20141208/CPRT-113-HPRT-RU00-HR83sa.pdf

4. December 4, 2014 Homework Assignment: Comments and feedback from the May 7, 2014 Certification Hearing Implementation, Certification, and Testing Workgroup January 15, 2015

5. Homework Assignment & Objective Assignment – Review the certification hearing testimony – Identify opportunities for improvement based on your own experiences – Come to January 15, 2015 (today’s call) meeting additional, potential ideas the workgroup could consider recommending to ONC to enhance the certification program Purpose – Review the feedback received in this assignment and during the May 2014 Certification Hearings to summarize lessons learned in order to have them clearly in mind as the workgroup receives and responds to the proposed Certification Rule 4

6. Overview of Comments Received Feedback received included observations and recommendations Topics 1.Interoperability 2.Usability and Workflow 3.Burden of Implementation 4.Guidance 5.Variation in Implementation 6.Parsimony 7.Testing Tools & Resources 8.Testing Procedures Out of Scope Comments – Several comments received related to topics outside of the scope of the Certification Program were not included 5

7. Observations Interpretation of the requirements have changed Certification efforts have shifted in focus from their original intent of driving adoption to an obligatory regulatory process The regulatory process seems to be driven by calendar rather than higher level objectives resulting in – Certification requirements and testing tools are not mature or complete at release – Lack of visibility into future requirements which leads to reduced engagement – Insufficient time to effectively implement 6

8. Recommendations Promote interoperability – Prioritize interoperability in certification criteria – Decrease specification on implementation and more on outcomes – Align various certification standards to support eCQM reporting – Extend interoperability focus and testing to include data exchange between acute care, ambulatory, and LTPAC Improve usability and workflow – Promote certification criteria that facilitates feedback loops – Acknowledge of industry product reviews (e.g., KLAS or others) – Require EHRs to undergo standardized usability test (e.g., http://www.measuringu.com/sus.php), at a minimum, publish those scores http://www.measuringu.com/sus.php – Focus certification requirements on core needs – Avoid requiring functionality that passes certification but isn’t useful to product consumers – Explore standardized workflows – Consider certification criteria to include workflow engine functionality 7

9. Recommendations Improve guidance – Improve audit guidance – Improve implementation assistance and guidance – Review guidance with all stakeholders prior to release to determine if any re-testing is required Reduce variation in implementation – Tighten certification specifications – Perform testing on real-world implementations Pursue parsimony – Minimize certification and attestation requirements to a few key priority areas – ONC needs to be parsimonious in terms of being prescriptive on what a vendor “must” do to meet certification 8

10. Recommendations Testing Tools & Resources Expand tools – Assure there are tools or systems in place to test all options available for EHs and EPs – Investigate the use of hubs prior to creating interoperability requirements that might entail significant costs – Include a rotating number of currently certified EMR/EHR vendors in pilots of new test procedures Automate and streamline – Increase automation of testing tools to publish thousands of test decks per “testable procedure” in an automated fashion that were consumable – Simplify the CHPL listing - condense the CHPL to show one listing with all of the items available in the version 9

11. Recommendations Testing Procedures Improve efficiency – Consider deeming for functionality that is already certified elsewhere (e.g., e-Rx tools) Increase clarity – Review certification and attestation requirements for ambiguity and address prior to releasing final requirements Enhance quality – Address data mapping issues prior to specification of code sets – Ensure accuracy, clinical validity and relevance, and efficiency of data entry to reduce time spent prepping the database for testing and time spent actually testing – Involve all stakeholders in creation of testing scripts – Pilot test scripts, methods, data sets, procedures, and testing tools with Use a sample of vendors of different size, scope, and architecture. 10

12. Recommendations Ease Burden of Implementation Prioritize and understand benefits – Minimize certification and attestation requirements to a few key priority areas – Understand the business case for each requirement Data use, beneficiary of data collection Improvements in efficiency or outcomes ROI Workflow Unintended consequences that might occur Consider level of effort and product development time – Assess the impact of new requirements on documentation burden and time impacts to enter, track, and report data – Allow adequate time for implementation – More time for vendors to develop safe and efficient products including the new functionality – Do not require all EHR users to have to upgrade in an unrealistically short time 11

13. Out of Scope Comments Broaden MU Incentive program to include long term care and post-acute care providers Consider incorporating telehealth and mobile Health encounters as part of MU to support access to care Quality control needs to be improved by those answering the CMS e-mail questions. We have numerous examples of completely opposite answers to the same questions Meaningful use objectives and quality measures must be aligned with each other, with certification requirements, and with clinical practice 12

14. Discussion 13

15. Certification Hearing Results May 7, 2014 December 4, 2014

16. Certification Hearing FACA member attendees 15 Paul Tang, MD, MS - Chair – Vice President, Chief Innovation & Technology Officer – Palo Alto Medical Foundation Michael H. Zaroukian, MD, PhD, FACP, FHIMSS – Co-Chair – Vice President & Chief Medical Officer – Sparrow Health System Carl D. Dvorak – Chief Operating Officer – Epic Systems Paul Egerman – Businessman/Software Entrepreneur Jennie Harvell PhD – Senior Policy Analyst – Department of Health & Human Services/ Office of Disability Aging & Long- Term Care Policy Joe Heyman, MD – Whittier IPA George Hripcsak MD, MS - Chair, Department of Biomedical Informatics – Director, Medical Informatics Services, Columbia University David Kates - Senior Vice President, Clinical Strategy - NaviNet Michael Lincoln - Veterans Health Administration Nancy J. Orvis, MHA, CPHIMS – Director, Business Architecture & Interoperability – Department of Defense Marc Probst – Vice President & Chief Information Officer – Intermountain Healthcare Donald W. Rucker, MD, MS, MBA – Associate Dean for Innovation – Ohio State University Wexner Medical Center Corporation John Travis, FHFMA, CPA – Senior Director & Solution Strategist, Regulatory Compliance – Cerner Corporation Charlene Underwood, MBA – Senior Director, Government & Industry Affairs – Siemens Medical Larry Wolf – Senior Consulting Architect – Kindred Healthcare

17. Hearing Purpose Understand what program features work well Assess challenges Identify opportunities to improve the program, leveraging lessons learned in stages 1 and 2 16

18. Hearing Panels Panel 1: Providers Panel 2: Vendors Panel 3: Certification/Accreditation Bodies Panel 4: Private Sector Representatives 17

19. Panel 1: Providers Ginny Lorenzi, NewYork-Presbyterian Hospital Chad Jensen, LaTouche Pediatrics, LLC John Berneike, Utah HealthCare Institute Colin Banas, Virginia Commonwealth University Health System Howard Hays, Indian Health Service Cletis Earle, CHIME Questions – Assuming we could design an ideal program, what is the benefit of having a certification program, from the perspective of your organization? How does a certification program help you? What are you looking for from a certification program? – Currently certification indicates that certain capabilities exist in an EHR, would certification ever indicate a level of quality? – What are the challenges you have experienced with the current certification program? – How would you design a certification program that would achieve the benefits you seek, while minimizing the burden to the participants? 18

20. Key points Panel 1: Providers EHR products may meet certification criteria, but the way the functions are implemented may disrupt workflow – Some functions fulfill the letter of the criteria, but not the intent (e.g., clinical summaries, patient education) – Some functions are implemented as “check the box” May be easy for vendor, but creates burden for provider and less useful Providers feel constrained to use products as certified, with inefficient work flow – Vendors be given enough flexibility to meet the rules without being constrained to a particular workflow Some certified products do not work, or do not work in all states 19

21. Key points Panel 1: Providers (II) Certification does not adequately cover interoperability Certification program should be less prescriptive; focus on what and less on how More flexibility and time for implementation is needed An ideal certification program would provide product comparisons in terms of their functionality 20

22. Panel 2: Vendors Mickey McGlynn, EHRA Sasha TerMaat, Epic Emily Richmond, PracticeFusion Joseph Geretz, SRSsoft Sarah Corley, NextGen Healthcare Systems Marc Probst, Intermountain John Halamka, Beth Israel Deaconess Medical Center Questions – Assuming we could design an ideal program, what is the benefit of having a certification program, from the perspective of your organization? – What are the challenges you have experienced with the current certification program? – How would you design a certification program that would achieve the benefits you seek, while minimizing the burden to the participants? 21

23. Key points Panel 2: Vendors Complete set of requirements are not provided with adequate time for development – Requirements change from what was originally defined in the certification rule impacting quality and usability Certification criteria for MU objectives, reports that measure these objectives, and the clinical quality measures are not aligned with each other and are not necessarily aligned with clinical practice The testing tools and associated data are not properly tested before they are rolled out for use in the vendor community, and change Recommend that the complexity of the program be reduced and that a Kaizen process be used to support an effective review of the certification program Focus certification on critical few elements (e.g., interoperability, CQMs) 22

24. Panel 3: Certification/Accreditation Bodies Amit Trivedi, ICSA Labs Kyle Meadors, Drummond Group - VIRTUAL Mark Shin, InfoGard Questions – Assuming we could design an ideal program, what is the benefit of having a certification program, from the perspective of your organization? – What are the challenges you have experienced with the current certification program? – How would you design a certification program that would achieve the benefits you seek, while minimizing the burden to the participants? 23

25. Key Points Panel 3: Certification/Accreditation Bodies Pilot test new procedures and test tools prior to publication Improve consistency between testing labs. Pilot tests should be a venue for all ATLs and ACBs to: – Observe testing to understand the expected results – Learn how the test tools operate – Provide feedback to ONC Testing tools need to be more automated to efficiently handle more: – Test cases, reuse test data sets, and; – Employ more robust types of testing methodologies including testing the security of products. Focus on certification criteria related to interoperability and security testing How EHRs handle various functionality should be left to developers to innovate 24

26. Panel 4: Private sector representatives Alisa Ray, CCHIT David Kibbe, Direct Trust Chris Carr, IHE Jitin Asnaani, CommonWell Mariann Yeager, Healtheway Questions – If you could design your ideal program, what would be the benefit of having a certification program, from the perspective of your organization? – What are the challenges you have experienced with the current certification program? – How would you design a certification program that would achieve the benefits you seek, while minimizing the burden to the participants? 25

27. Key Points Panel 4: Private sector representatives Need additional up-front testing and quality assurance Mid-cycle revisions are disruptive to the overall program Need subject matter experts in program development Enhanced collaboration between the private sector and the federal government would help Focus on critical few 26

28. Overall Summary No disagreement about intent of MU objective Insufficient time for product development and testing Concerns about certification include: – Specificity of certification criteria locking in vendor-created inefficient workflows for providers – Incompletely tested and unstable testing tools delays certification and creates rework – Inconsistent interpretations among ATLs, ACBs, and auditors – Certification does not guarantee integrated product or interoperability – No clearinghouse for timely feedback and response (i.e., need faster than NPRM cycles) Time required for certification (or documenting certification) crowds out innovation 27

29. Final Recommendations (I) Kaizen Motivation: current process is inefficient and burdensome Objective: Need a coordinated, integrated, well understood certification process with minimum of burden Recommendation: Kaizen covering end-to-end certification process from translation of MU objective to certification criteria to development of testing scripts to development (and QA) of testing tools to conduct of test to auditing Involve broad stakeholders from providers to developers to ATLs/ACBs to auditors Establish certification roadmap and timelines Create a timely PDCA mechanism for feedback and continuous improvement 28

30. Final Recommendations II Focus Motivation: providers and vendors (and ACBs) feel overwhelmed by the pace of changing requirements Objective: Focus on critical few Limit scope of certification to: – Interoperability – CQMs – Privacy and security These require – Cross-organizational collaboration (and policy interoperability) – Alignment of standards, measures, and programs – Overarching governance – Public-private collaboration 29