National University Of Rwanda Faculty Assessment: Self Administered Questionnaires Inis Jane Bardella, MD, FAAFP L. Miriam Dickinson, PhD Mark Hotchkiss Doug Fernald, MA Calvin Wilson, MD N. Michelle Shiver, BSE Center for Global Health University of Colorado Denver

Background USAID funded Rwanda Medical Training Initiative – RMTI 3 Partners – UCD NUR Centura Health Three year project, Two Goals  Assist NUR in the sustainable development of current post-graduate programs  Assist NUR in the development and implementation of a sustainable post-graduate program in Family Medicine

Background 4 of 9 project Objectives require faculty development interventions for achievement.  Assist faculty in mastering the clinical, teaching, research and management skills necessary for medical education.  Assist in the development and implementation of processes for feedback, evaluation and improvement of medical knowledge, clinical competencies and patient interactions of learners.  Assist in the identification and ongoing development of faculty for the training of family medicine post-graduates.  Assist in the development and implementation of processes for evaluation of medical knowledge, clinical competencies and patient interactions of [family medicine] learners.

Research Objectives Multiphase project to determine faculty needs and behaviors pre-post faculty development interventions Aim of Phase One: Determine baseline faculty needs and behaviors in clinical care, teaching, research and management. Presentation focused on the teaching needs and behaviors.

Methods Semi-structure interviews with NUR administration and academic chairs – March 2007 Self Administered Questionnaires – SAQs  Determine individual faculty develop needs  Guide program development  Address project objectives SAQ format and content  Semi-structured interview data  Literature review  PI and project group experience

Methods 8 Page questionnaire  Demographics and job responsibilities  Clinical, teaching, research and management abilities and needs  Learning preferences for faculty development 28 Open-ended questions – Qualitative 30 5-point Likert scale items – Quantitative Translated into French IRB review and approval for Exempt Status

Methods July 2007 Faculty, clinical heads and senior instructors were recruited during focus groups SAQs distributed by and returned to PI French responses translated into English Data entered into an Excel database Descriptive statistics – SAS vs Excel Qualitative responses grouped by theme Comparisons of individuals and between groups

Results Response rate 42% - 8/19 eligible faculty  5/6 (83%) CHUK  3/6 (50%) King Faisal  0/7 (0%) CHUB IM, ob/gyn, peds and surgery represented High percentage with teaching responsibilities  Medical student education – 75%  Post-graduate student education – 87.5% Professional satisfaction and years in clinical practice were high.  Median 4 (1 none, 5 high)  24.5 median years, range 14-33

Results: Teaching Barriers to becoming the ideal teacher  Lack of methodology sessions to improve teaching  Lack of teaching materials/resources  Heavy work load  Poor motivation/incentives for becoming a full-time teacher Help you to become the ideal teacher  Training, seminars in teaching methodologies  Access to teaching resources  Time

Results: Teaching Barriers to Effective Teaching Median Standard Deviation Insufficient time available to teach Lack of ready access to reference materials Insufficient communication from med school/post grad program Support from colleagues/partners point Likert Scale, 1=Not a problem, 5=Major Problem

Use of Teaching Techniques Median Level of Use Standard Deviation Interactive Discussion Didactic Teaching Direct Observation Providing Feedback Identifying student educational objectives point Likert Scale, 1=Never heard, 5=Use several times/day

Methods for Evaluation of Learners Median Level of Use Standard Deviation Mid-course learner meeting/review End-of-course learner meeting Chart review Written exam Oral exam Behavior ratings OSCE point Likert 1=Never heard, 5=Use with almost all, >75%

Methods with Learners Exhibiting Difficult Median Proficiency Standard Deviation Identify learner having difficulty Deliver negative feedback Develop an action plan for correction of difficult Re-evaluate the learner Accurately evaluate the learner having difficulty point Likert Scale, 1=None, 5=High

Results: Faculty Development Most useful faculty development topics  Teaching methodology  Evaluation techniques  Seminars on medical research

Format Preference for Faculty Development Median Standard Deviation PDA51.51 Online, web based Live, group Audio, CD Video, DVD point Likert Scale, 1=Never use, 5=Use often, prefer

Conclusions Faculty development strategies which address teaching and evaluation methods are desired and needed. Access to education resources is desired and needed. Time and communication are barriers to education even in the developing world. The self-reported level of use of interactive teaching techniques is high.

Conclusions Traditional methods are used to evaluate learners. While the proficiency in identifying learners exhibiting difficulty is high, proficiency with techniques to address the difficulty are low to moderate. PDAs, web-based and live group formats are preferred for faculty development programs.

Limitations Small sample size No response from NUR based faculty Self-assessment was a novel concept. Some terminology did not translate well. –Voluntary –Education concepts –Likert scale format Translation into French not preformed by a medical educator or researcher. Did not do two-way translation.

Implications The self-reported high use of interactive teaching, moderate level of proficiency with difficult learners and preference for PDA programs were unexpected and need to be explored. Needs are similar to the developed world. Faculty want to be effective regardless of country or continent. Our lessons learned can be used for more effective international needs assessment. This is a model for international faculty needs assessment.

Murakoze!

Results: Clinical Barriers to becoming the ideal clinical doctor  Heavy clinical load  Limited time  Lack of CME/CPD  Lack of proper laboratory services/equipment  Lack of referral systems/subspecialists  No support for career development Help you to become the ideal clinical doctor  CME/CPD/education resources  Study tours/exchanges at hospitals  Reduce the workload/Employ more doctors  Increase the number of collaborators