1. National Immunization Partnership with the APA: A Quality Improvement Project Specific to HPV Vaccination (NIPA) CDC Grant # 1H23IP000950

2. AGENDA Introductions 5min What to expect from this project… Project overview 15min Timeline 5min Monthly Collaborative 10min Study Materials 15min Questions? 10min

3. NIPA CORNET QI Team Monthly Call Faculty: –Cynthia Rand, MD, MPH –Paul Darden, MD –Stanley Schaffer, MD, MS –William Stratbucker, MD, MS –Janet Serwint, MD Other Project Faculty: –Peter G. Szilagyi, MD, MPH –Sharon Humiston, MD, MPH

4. Project Goals Develop a sustainable strategy to train residents and faculty about optimal adolescent immunization methods Contribute to increased rates of HPV vaccination Provide excellent QI training

5. Background & Context Studies show that what residents learn during training informs how they later practice Teaching residents skills and approaches can improve adolescent immunization delivery There are approximately 200 Pediatric residency training programs in the US Pediatric faculty need to be trained in order to provide effective training to residents The best training occurs in the context in which care is delivered, i.e. immunization delivery in clinic.

6. Specific Aims Develop and execute a national QI project addressing adolescent immunizations with a focus on HPV Identify and test strategies to improve health care delivery of HPV vaccines to adolescent males and females

7. QI Practice Change Strategies All sites incorporate Strong Provider Recommendations Practices choose which other QI strategies to use: Prompts to providers to give vaccine at the time of the visit; specifically prompts by nurse/staff at ALL visits (including acute visits) Standing Order protocols that empower office personnel to vaccinate, if a patient is due, without direct doctor involvement at the time of patient care at ALL visits Reminder Recall systems to notify members of the target population that vaccinations are due or overdue Feedback from monthly chart audits to assess process measures

8. Faculty Responsibilities Obtain IRB approval or equivalent letter for a quality improvement (QI) project from your institution Participate in monthly learning collaborative teleconferences over 9 month project. Faculty must attend at least 6 calls to receive MOC credit. Work with residents to pull baseline chart review audits for baseline data (50 charts) Work with residents to pull interim chart reviews and reports for monthly QI Learning Collaborative (10 audits per month) Each site asked to keep a running record of practice changes and process assessments Learning Collaborative Plan

9. ACGME: Practice-based Learning & Improvement Residents must demonstrate the ability to investigate and evaluate their care of patients, to appraise and assimilate scientific evidence, and to continuously improve patient care based on constant self-evaluation and life-long learning. Resident Learning Requirement:

10. MILESTONES: -Practice based learning and improvement -Systematically analyze practice using quality improvement methods, and implement changes with the goal of practice improvement MOC for Residents: -Residents can now earn Part 4 MOC credit during residency for meaningful participation in QI activities -Resident MOC credit will be “in the bank” for when they become certified. It will then be applied to their first MOC cycle Resident Learning Requirement (cont’d):

11. Resident Responsibilities View/Review ALL orientation and training podcasts Participate in chart reviews:  Review at least 10 charts of pre- chart audit  Review at least 10 charts of post- chart audit  Review at least 20 charts of monthly audits (2 months) Participate in at least 6 phone calls or review recordings Participate in review of data analysis Brainstorm with clinic QI team as to how to make practice change most effective Learning Collaborative Plan

12. Benefits to Participating Receive MOC credit after meeting all requirements (faculty and residents) Participate in a national QI project for faculty and residents Enhanced education in QI principles Participate in a scholarly project Receive own site-specific data

13. Characteristics of CORNET Sites 2015- 107 pediatric training programs (54%) 130 clinical practice sites Mean of 52 residents/program (16-150) > 5500 pediatric residents Care for > 851,890 pediatric patients

14. Partnership with NIPN NIPN: National Improvement Partnership Network Based at University of VT A network of over 20 states that have developed Improvement Partnerships to advance quality and transform healthcare for children and their families

15. Characteristics of NIPN Sites 2015- 6 State IPs: Alabama, Maine, New Hampshire, New Jersey, Tennessee, Vermont An Improvement Partnership (IP) is a durable state or regional collaboration of public and private partners that uses measurement-based efforts and a systems approach to improve child health outcomes and the quality of their healthcare.

16. NIPA Participation Map CORNET – 16 programs (blue markers) NIPN – 6 states (green markers)

17. May - June 2015 Enrollment Phase Program Selection Notification Contact Information Form IRB QI proposal submission July - August 2015 Orientation Phase Orientation Call – Project Overview/Data Collection Practice Readiness Assessment/Office Systems Inventory Begin Baseline Chart Reviews (50) QI Module Training Strong Provider Recommendation Training August 2015 - April 2016 Learning Collaborative Implementation Phase Monthly Learning Collaborative Calls Monthly PDSA Self Assessments Monthly Chart Reviews (10/month) NIPA Staff Impact Survey (Midpoint) May 2016 Wrap Up Phase Begin Post LC Chart Reviews (50) NIPA Staff Impact Survey (Endpoint) Office Systems Inventory (Endpoint) MOC attestation Project Timeline YOU ARE HERE

18. Responsibilities of Site Participants PRE-Implementation of Practice Change May - June 2015 Enrollment Phase Program Selection Notification Contact Information Form IRB QI proposal submission July - August 2015 Orientation Phase Orientation Call – Project Overview/Data Collection Practice Readiness Assessment / Office Systems Inventory Begin Baseline Chart Reviews (50) QI Module Training - 4 parts ~60 minutes Intro., Leading change, AIM statements, QI measures Strong Provider Recommendation Training ~10 minutes

19. Responsibilities of Site Participants PRE-Implementation of Practice Change Participate in a project orientation call  Obtain IRB approval from each institution Complete the Practice Readiness Assessment* Complete the Office Systems Inventory* Complete training modules by August 7 2015 Obtain pre data to establish baseline- review a total of 50 charts to be completed by September 1 2015* If you haven’t already done so, please send in your denominators for randomization *Data entry online to be completed via REDCap. A site specific link will be provided to each site

20. Responsibilities of Site Participants Implementation of Practice Change Important dates: Monthly Data due: 5 th day of the month Monthly Calls scheduled: 4 th week of the month August 2015 - April 2016 Learning Collaborative: - Monthly Learning Collaborative Calls - Monthly PDSA Self Assessments: Barriers encountered and lessons learned -Monthly Chart Reviews (10/month) for process measures: missed opportunities -Staff Impact Survey (Midpoint)

21. Responsibilities of Site Participants Post Implementation of Practice Change MOC attestation: Use MOC Tracking Log in Virtual Toolkit Instructions will be sent towards the end of the series of LC calls May 2016 Wrap Up Phase Begin Post-LC Chart Reviews (50) for outcome measures: HPV vaccination rates Post- Office Systems Inventory Staff Impact Survey (Endpoint) MOC attestation

22. Patient Population Baseline (50) and post intervention (50) chart audits: All patients age 11-17 years, 25 males/25 females Any visit reason (health maintenance, chronic disease, acute care) in 9 month timeframe Include all visits, including those up-to-date with HPV vaccination (different than monthly) Follow randomization scheme Monthly (10 charts/month) Patient age 11-17 years Any visit type in the prior month ONLY IF Eligible for HPV vaccine at the visit (different than baseline)

23. QI Project Progress –to date February 2015  MOC Part IV Credit approved  Recruitment of CORNET Practices and NIPN State IPs June 2015  IRBs approved at PI sites  31 CORNET programs expressed interest  Wave 1 participants notified (16 programs)

24. QI Project –NEXT STEPS  Sites to have completed:  Contact Information Form  Practice Readiness Assessment  Office Systems Inventory  IRB QI applications at respective sites  Send in denominator for randomization  Review of QI training modules and Strong Provider Recommendation module  CORNET Sites to begin Learning Collaborative calls:  August 2015

25. Learning Collaborative Plan Tools to help you navigate Project Timeline A timeline to help move you through the project Listserv address A way to communicate with your fellow project colleagues NIPA_QI@listserv1.academicpeds.org NIPA_QI@listserv1.academicpeds.org Virtual Toolkit An easy way to reference all your project materials www.academicpeds.org/NIPA www.academicpeds.org/NIPA REDCap for all data entry Secure link to enter data (site specific)

26. All data entered electronically via REDCap All data and survey tools at the practice-level, not the provider level Main faculty contact receives email with practice-specific links to data tools Main contact shares submission links with practice team and coordinates data entry Do not send any forms or medical record information to the NIPA project team Data Submission

27. REDCap Tips Most fields are required: cannot move to next page without completing all required fields Can save partially completed data and return later to complete: SAVE RETURN CODE Review data tools on the Virtual Toolkit – www.academicpeds.org/NIPA - Data Collection www.academicpeds.org/NIPA

28. Distributed: Pre-project Purpose: Assess practice experience and comfort with QI to guide practice coaching Practice Readiness Assessment

29. Distributed: Pre- and post-project Purpose: Describe practice’s current systems for delivery of HPV vaccine and identify potential areas for system change Discuss questions with practice team Office Systems Inventory

30. Distributed: Pre- and post-project* Purpose: Calculate rates of missed opportunities, HPV series initiation and completion Pre- & Post- Intervention Chart Audit 50 charts pre- and 50 post- 25 males 25 females *Please send denominator if you haven’t already

31. Eligible patients: Adolescents 11-17, seen for any visit type between August 1, 2014 and April 30, 2015 Randomization procedure Practice generates list of all patients seen between 8/1/2014 and 4/30/2015 Birthdates between 5/1/1997 and 8/1/2003 Alphabetize patient list and number them 1,2,3… Email total number of patients (NOT patient list) Statistician generates random list of numbers corresponding to patients on site patient list Patient charts reviewed in that random order Pre-Intervention Chart Audit

32. Chart Audit Patient Log tracks: Randomization Scheme Gender and Medical Record Number REDCap Patient ID Calculator function helps you with validation: Calculate patient age at visit dates Calculate HPV vaccine eligibility dates Track # of males and females Pre-Intervention Chart Audit

33. Remember to record REDCap Patient ID in the Patient Log NIPA team will reference this number if there are any questions about the data Pre-Intervention Chart Audit

34. Distributed: each month of Learning Collaborative Purpose: Assess progress and evaluate system changes 10 charts per month Convenience sample: not randomized – suggest 5 males + 5 females from different weeks, days and providers All sites: evaluate missed opportunities on monthly basis Reminder Recall: select 10 patients contacted via reminder-recall system, audit whether they presented for a visit that month Monthly Chart Audit

35. Baseline/post chart reviews = Patient Level EVERY visit in the time frame, whether eligible for HPV vaccine or not Monthly chart reviews = Visit level ONLY visits where patients were eligible to receive HPV vaccine Monthly Chart Audit

36. Distributed: each month of Learning Collaborative Purpose: Reflection on Plan-Do-Study-Act cycle Record successes and barriers you encounter, examine the progress you have been making through the strategy you selected, and make changes that will lead to maximal improvement of your immunization rates Discussed on monthly LC calls PDSA Self-Assessment Tool

37. Monthly Data Report

38. Distributed: mid- and post-project Purpose: Balancing measure to evaluate whether the time and effort required to implement office systems interventions affect staff in planning, scheduling, and conducting visits for adolescents who are due/overdue for HPV vaccine Staff Impact Survey

39. Any questions? Please contact : Nui Dhepyasuwan, MEd Phone: 703/556 9222 x103 I Fax: 703/556 8729 nui@academicpeds.org Beth King, MPP Phone: 703/556 9222 x140 I Fax: 703/556 8729 beth@academicpeds.org Listserv: NIPA_QI@listserv1.academicpeds.orgNIPA_QI@listserv1.academicpeds.org