Assessment of Initial Clinical Competence for NP Regulation: The Development of an Objective Structured Clinical Examination Marie Napolitano RN, PhD, FNP Oregon Health & Sciences University Chief Examiner, NP OSCE College of Registered Nurses of BC Alison Roots RN, MHSM Project Manager, NP OSCE College of Registered Nurses in British Colombia

Innovation First OSCE for NP regulation (licensure) in the international community First OSCE for NP regulation (licensure) in the international community

OSCE Objective Structured Clinical Exam Objective Structured Clinical Exam Used in Canada by the disciplines of medicine, physiotherapy, midwifery and pharmacy to assess clinical competency Used in Canada by the disciplines of medicine, physiotherapy, midwifery and pharmacy to assess clinical competency Structured examination that biopsys or samples a variety of clinical skills Structured examination that biopsys or samples a variety of clinical skills

College of Registered Nurses of British Colombia Responsible for the licensure or registration of nurse practitioners in the province of British Colombia (BC), Canada Responsible for the licensure or registration of nurse practitioners in the province of British Colombia (BC), Canada Received approval of requirements for NP licensure in 2005 – same time as new NP programs were graduating first NPs in BC Received approval of requirements for NP licensure in 2005 – same time as new NP programs were graduating first NPs in BC

Licensure for NPs in BC Requirements Requirements Graduation from a CRNBC recognized NP program or equivalent Graduation from a CRNBC recognized NP program or equivalent Pass score on an approved written exam Pass score on an approved written exam Pass on the OSCE Pass on the OSCE

Objectives of the OSCE Assess the capacity of NP Candidates to provide health care services to a client from a holistic nursing perspective Assess the capacity of NP Candidates to provide health care services to a client from a holistic nursing perspective To determine whether the minimum standard to practice has been demonstrated To determine whether the minimum standard to practice has been demonstrated To assess the candidates readiness for independent safe practice To assess the candidates readiness for independent safe practice

Development of the OSCE Official parties identified Official parties identified Project manager Project manager Nurse Practitioner Examination Advisory Group Nurse Practitioner Examination Advisory Group Chief Nurse Examiner Chief Nurse Examiner Case (scenario) authors Case (scenario) authors Examiners Examiners

Development of the OSCE Writing case scenarios Writing case scenarios Creating checklists - observable behaviours specific to case scenarios Creating checklists - observable behaviours specific to case scenarios Determining checklists weighted scores and minimum passing scores Determining checklists weighted scores and minimum passing scores Identifying critical incidents Identifying critical incidents Developing the global assessment Developing the global assessment Training standardized patients (SPs) Training standardized patients (SPs) Orienting examiners Orienting examiners Orienting candidates Orienting candidates

OSCE components Five minute scenarios – client interactive Five minute scenarios – client interactive PEPs (post exam probes) – written stations to accompany five minute scenarios PEPs (post exam probes) – written stations to accompany five minute scenarios Ten minute scenarios – client interactive Ten minute scenarios – client interactive

Focus of testing Integration of holistic nursing approach with the clinical skills of diagnosing, managing and prescribing Client Encounters Client Encounters complete history or complete history or perform physical exam or perform physical exam or Counsel / manage Counsel / manage PEPS PEPS differential diagnosing, diagnostic tests, management plans, writing prescriptions or referral letters. differential diagnosing, diagnostic tests, management plans, writing prescriptions or referral letters.

Examination Competencies Assess and diagnose clients health and illness status Assess and diagnose clients health and illness status Manage client health care Manage client health care Promote health and prevent illness/injury Promote health and prevent illness/injury Maintain professional responsibility and accountability Maintain professional responsibility and accountability

Case scenarios for FNP Cover the lifespan Cover the lifespan Include common client problems and needs including prenatal & post natal Include common client problems and needs including prenatal & post natal Developed by experienced NPs Developed by experienced NPs

Global assessment rating scale Relationship development Relationship development Clinical knowledge and competence Clinical knowledge and competence

Candidates Graduates of recognized NP programs in BC Graduates of recognized NP programs in BC Graduates from programs that are educationally equivalent to BC programs Graduates from programs that are educationally equivalent to BC programs Candidates determined to be educationally equivalent by Prior Learning Assessment and Recognition Candidates determined to be educationally equivalent by Prior Learning Assessment and Recognition

Examiners Role Role Criteria to be examiner Criteria to be examiner Faculty in Masters level NP program or Faculty in Masters level NP program or Experienced NP preceptor Experienced NP preceptor Need to use experienced NPs from Washington and Oregon to begin Need to use experienced NPs from Washington and Oregon to begin

Examination Scoring Process Passing score on a minimum number of client interactive stations (scenarios) Passing score on a minimum number of client interactive stations (scenarios) Minimum overall competency score Minimum overall competency score Demonstrate no critical incidents Demonstrate no critical incidents

Examination Advisory Group Comprised of: Comprised of: Chief and Deputy Nurse Examiners, FNP from BC (passed OSCE on first run), Registered Nurses with education and evaluation specialties Chief and Deputy Nurse Examiners, FNP from BC (passed OSCE on first run), Registered Nurses with education and evaluation specialties Role of the Examination Advisory Group Role of the Examination Advisory Group Determine OSCE scoring and pass criteria Determine OSCE scoring and pass criteria Review overall performance of scenarios Review overall performance of scenarios Review performance of candidates Review performance of candidates

Scoring Development Process Pre- exam minimum passing standard set for checklist and PEP Pre- exam minimum passing standard set for checklist and PEP Scenario field tested ( trialed) Scenario field tested ( trialed) Post field test – performance of scenario analyzed, minimum passing score reviewed Post field test – performance of scenario analyzed, minimum passing score reviewed Checklist, PEP questions & passing scores revised as needed Checklist, PEP questions & passing scores revised as needed

Changes since first trial run Scenarios fine tuned Scenarios fine tuned Reliability and validity improved Reliability and validity improved

Results of OSCE At this time, minimal number of candidates does not allow for release of results At this time, minimal number of candidates does not allow for release of results

Factors that play important role in success Orientation to examination process Orientation to examination process Candidate preparedness Candidate preparedness

Future directions Addition of adult and pediatric NP examinations in 2006 Addition of adult and pediatric NP examinations in 2006 Use of Canadian NP examiners, case writers Use of Canadian NP examiners, case writers

Thank you