Recertification: Aligning your options with your purpose Andrew Dallas, NCCPA Timothy Muckle, NBCRNA Chad Buckendahl, Alpine Testing

Overview of Session Defining recertification Purpose Program design considerations Sources of evidence Methods/strategies for evidence collection Evaluating evidence 2

Defining recertification Continued competency Enhanced competency Purpose Public protection Knowledge and skill acquisition/refinement Continuous learning Credibility of the credential Voluntary vs. mandatory credentials 3

Program Design Sources of evidence Continuing education Committee contributions Journal articles, book chapters, conference presentations Retesting Methods/strategies for evidence collection Portfolios Online repositories, documentation systems 4

Program Design Evaluating evidence Defining review criteria Determining relative weighting for systems with multiple measures Incorporating an audit function Equivalence of expectations Aligning expectations with the purpose of your program 5

Andrew D. Dallas, PhD Psychometrician Recertification of Physician Assistants

Background on the organization and the profession What the NCCPA has done in the past Changes the NCCPA has made recently Issues/Topics that will affect further changes Presentation Outline

About NCCPA Independent, not-for-profit organization Current Mission: To serve the public through exemplary programs that evaluate critical PA competencies and require the pursuit of life-long learning and improvement Previous Mission: To ensure that certified PAs meet professional standards of knowledge and skills Over 105,000 PAs have been certified since NCCPA was established in 1975

Physician Assistants are an extremely flexible profession in the healthcare industry. Initially set out to be a Primary Care profession (Family & General Practice/Internal Medicine/ General Pediatrics) but now ~70% practice in non-Primary Care specialties. Certification is required for initial licensure in all fifty states. (Continued certification required in ~half of the states; however, employers can also mandate continued certification). A little about Physician Assistants (PAs)

If 70% of all PAs are in a non-primary care specialty, should there be different expectations for PAs to recertify? The majority (if not all) of the changes to recertification have been in an effort to address the specialization of the profession. One thorny psychometric/practical issue

The recertification exam Equivalency vs. Equal All PAs – regardless of specialty – recertify by passing this exam. PANRE

Pathway II was one of the first major changes to the recertification process. Started in the late 1990s as an alternative to PANRE. PAs could choose Pathway II or PANRE. Discontinued in Take home examination and also included requirements for additional professional/educational activities. Thought was that specialists would take the Pathway II. (Analysis of the testing years, showed a rate of 69% non-PC for Pathways II, 61% non-PC for PANRE). Pathway II

PANRE changed to include two components: 60% primary care, 40% practice focused. Blueprint is the same for all exams. Practice focused component consisted of three options: Primary Care (more of the same) Adult Medicine (removed the Pediatrics from the 40% but not the exam). Surgery (General Surgery, not necessarily appropriate for specialized surgery PAs). Thus far, Primary Care is still the most popular of the three options. Practice-focused PANRE

Recertification occurred every 6 years. Recertification exam (PANRE) was one part of the entire recertification effort. CME was required every 2 years between PANRE attempts. Category I CME: 50 hours. (Formally planned activities) Category II CME: 50 hours. (Self-reported activities) The [not too distant] Past

Big changes to the recertification program started in Recertification occurs every 10 years. CME is required every 2 years between PANRE attempts Category I CME : 50 hours Category II CME : 50 hours 20 of the Category I hours must be designated as “self- assessment”(SA-CME) and/or performance improvement CME (PI-CME). Today

What is Self-Assessment (SA)? Self-assessment is the process of conducting a systematic review of one’s own performance, knowledge base or skill set for the purpose of improving future performance, expanding knowledge, or honing skills.

Why Self-Assessment (SA)? 2012 study concluded that more than 20% of core information guiding clinical practice is changed within one year based on new evidence or guidelines. Self-assessment makes CME more meaningful and practical to one’s practice. It requires active engagement in the learning process.

Approved for 20 AAPA Category 1 Self-Assessment CME credits 400 clinical vignette based questions Topics derived from the NCCPA Content Blueprint Comprehensive explanation of the correct and incorrect answer choices A follow up One Step Further question to reinforce the content just tested Performance analysis to track strengths and weaknesses Cumulative 50% correct to receive credit Cost: $79

What is Performance Improvement (PI) CME? PI-CME is active learning and the application of learning to improve your practice. Three-step process: 1.Compare some aspect of practice to national benchmarks, performance guidelines or other established evidence-based metric or standard. 2.Based on the comparison, develop and implement a plan for improvement in that area. 3.Evaluate the impact of the improvement effort by comparing the results of the original comparison with the new results or outcomes.

What is Performance Improvement (PI) CME? Put another way, there are three stages for which you can earn CME credit: Stage A - Identify evidence-based measure and assess practice (5 PI-CME credits) Stage B - Intervention(s) (5 PI-CME credits) Stage C - Re-measure; document improvement (5 PI-CME credits) Completing all 3 stages (bonus of 5 PI-CME credits)

METRIC is offered by the American Academy of Family Physicians Can be completed individually or as group Cost to PA non-members: $125 Each module = 20 PI-CME credits Modules available in Diabetes Asthma Hypertension Geriatrics

Additional credential for specialty practitioners. Voluntary program, no effect on recertification. Must be renewed on a similar 10 year cycle. CAQs

What Specialties? Emergency Medicine Cardiovascular & Thoracic Surgery Orthopaedic Surgery Nephrology Psychiatry Exam Debuted in 2014: Pediatrics Hospital Medicine

Further consideration of the issue of specialization. Is practice-focused PANRE working as we intended it to do so? Empirical analysis of the changes to brand new recertification policies. Do the new types of CME engender a stronger PA workforce and better outcomes for patients? Issues which remain

Recertification programs should never be written in stone. The NCCA standards are not specific on what has to exist; there is room for different models. Continued research should help inform decisions made on recertification. Conclusions

Tim Muckle, PhD Director, Testing Services Recertification of Nurse Anesthetists

A little bit about nurse anesthesia, and Environment in which the NBCRNA operates Current recertification requirements Comprehensive revision resulting in Continued Professional Certification (CPC) Program Issues related to testing and psychometrics Topics

Advanced practice nurses Highly critical field Personality type ‘A’ Highly charged political environment Nurse Anesthesia

Current, unrestricted license Practice requirement – 850 hours 40 hours continuing education Affidavits Fee Renew every 2 years This is has been the recertification model since Current Recertification Reqs

Why Change? So… Why Change?

Changing certification requirements/healthcare trends Evolving knowledge and skills Lifelong learning throughout your career Technology provides new opportunities Gain credit for activities nurse anesthetists already do

Comparisons Recertification Components Nurse Anesthetists (Current) Anesthesiologist Assistants Anesthesiologists Certified Nurse Midwives Nurse Practitioners (AANP) Nurse Practitioners (NCC) FrequencyEvery 2 years Every 10 yearsEvery 5 years Every 3 years Current Licensure YesNAYes Continuing Education 40 hours 350 hours 20 hours 3 Self Learning Modules 75 hours hours ExaminationNA Yes Every 6 years Yes 7 to 10 of each Cycle ProposedOptionalYes Practice850 hoursNot requiredYesNot required1,000 hoursNot required

So…Why Change? According to research conducted by the Citizen Advocacy Center in 2007, consumers: expect healthcare providers to be competent throughout their professional careers. believe that professional regulatory bodies ensure licensees' competency.

So…Why Change? 52% believe being licensed means periodic evaluation and assessment. 95% believe healthcare professionals should be required to show up-to-date knowledge as a condition of re-licensure. 90% believe it is important for healthcare professionals to be periodically re-evaluated. 84% believe healthcare professionals should be evaluated on their qualifications.

Shift to Continuing Competency Initial certification is only a start Knowledge gained at one point in time is no longer enough Competency demonstrated through evidence-based learning and assessment Professional growth and development must be lifelong and integrated into practice Credentialing organizations and other advanced practice nursing professions have made the change

New and Improved!

CPC Program 8 years, comprising two 4- year CPC Cycles Class A Credits, Assessed CE Class B Credits, Professional Development Core Modules CPC Examination RN Licensure and Active Practice

Psychometric Issues Class A: Assessed CE How does NBCRNA influence quality of assessments? Reliability: 10 questions per credit hour Quality item writing? Competency modules Require vendors to report assessment data, reliability, item statistics, item revision and replacement Cut scores

Examination Airway management techniques (28%) Applied clinical pharmacology (28%) Human physiology and pathophysiology (28%) Anesthesia technology and equipment (16%)

Examination – Psychometric Issues Pass/Fail is phased in over 20 years First administration will be “Diagnostic”. If examinees do not meet “performance standard,” they do not lose certification, but are required to complete additional CE in identified core areas

Examination – Psychometric Issues This has implications for: Test specifications (mainly, length) Standard setting

Examination – Psychometric Issues “The CPC examination is NOT the same examination you took to first become a CRNA. Rather, it will be based on “the expert knowledge and clinical judgment gained through practice.”

Examination – Psychometric Issues Implications for Item Design How do we create a test that is: Distinct in format from certification exam Clinically applied Not focused on minutiae and esoteric knowledge Clinical simulation? Items based on clinical scenarios?

Examination – Psychometric Issues CPC will be largest testing program Reconsider item development models Will test be subject to heightened risk of compromise? What’s the right number of items to develop? Enhance Test Security Practices, P&P Test security audit