1. National University Of Rwanda Faculty Assessment: Self Administered Questionnaires Inis Jane Bardella, MD, FAAFP L. Miriam Dickinson, PhD Mark Hotchkiss Doug Fernald, MA Calvin Wilson, MD N. Michelle Shiver, BSE Center for Global Health University of Colorado Denver

2. Background USAID funded Rwanda Medical Training Initiative – RMTI 3 Partners – UCD NUR Centura Health Three year project, 2007-2009 Two Goals  Assist NUR in the sustainable development of current post-graduate programs  Assist NUR in the development and implementation of a sustainable post-graduate program in Family Medicine

3. Background 4 of 9 project Objectives require faculty development interventions for achievement.  Assist faculty in mastering the clinical, teaching, research and management skills necessary for medical education.  Assist in the development and implementation of processes for feedback, evaluation and improvement of medical knowledge, clinical competencies and patient interactions of learners.  Assist in the identification and ongoing development of faculty for the training of family medicine post-graduates.  Assist in the development and implementation of processes for evaluation of medical knowledge, clinical competencies and patient interactions of [family medicine] learners.

4. Research Objectives Multiphase project to determine faculty needs and behaviors pre-post faculty development interventions Aim of Phase One: Determine baseline faculty needs and behaviors in clinical care, teaching, research and management. Presentation focused on the teaching needs and behaviors.

5. Methods Semi-structure interviews with NUR administration and academic chairs – March 2007 Self Administered Questionnaires – SAQs  Determine individual faculty develop needs  Guide program development  Address project objectives SAQ format and content  Semi-structured interview data  Literature review  PI and project group experience

6. Methods 8 Page questionnaire  Demographics and job responsibilities  Clinical, teaching, research and management abilities and needs  Learning preferences for faculty development 28 Open-ended questions – Qualitative 30 5-point Likert scale items – Quantitative Translated into French IRB review and approval for Exempt Status

7. Methods July 2007 Faculty, clinical heads and senior instructors were recruited during focus groups SAQs distributed by and returned to PI French responses translated into English Data entered into an Excel database Descriptive statistics – SAS vs Excel Qualitative responses grouped by theme Comparisons of individuals and between groups

8. Results Response rate 42% - 8/19 eligible faculty  5/6 (83%) CHUK  3/6 (50%) King Faisal  0/7 (0%) CHUB IM, ob/gyn, peds and surgery represented High percentage with teaching responsibilities  Medical student education – 75%  Post-graduate student education – 87.5% Professional satisfaction and years in clinical practice were high.  Median 4 (1 none, 5 high)  24.5 median years, range 14-33

9. Results: Teaching Barriers to becoming the ideal teacher  Lack of methodology sessions to improve teaching  Lack of teaching materials/resources  Heavy work load  Poor motivation/incentives for becoming a full-time teacher Help you to become the ideal teacher  Training, seminars in teaching methodologies  Access to teaching resources  Time

10. Results: Teaching Barriers to Effective Teaching Median Standard Deviation Insufficient time available to teach 41.15 Lack of ready access to reference materials 41.13 Insufficient communication from med school/post grad program 3.51.06 Support from colleagues/partners 3.51.28 5-point Likert Scale, 1=Not a problem, 5=Major Problem

11. Use of Teaching Techniques Median Level of Use Standard Deviation Interactive Discussion 40.99 Didactic Teaching 41.52 Direct Observation 40.83 Providing Feedback 31.03 Identifying student educational objectives 31.35 5-point Likert Scale, 1=Never heard, 5=Use several times/day

12. Methods for Evaluation of Learners Median Level of Use Standard Deviation Mid-course learner meeting/review 2.50.5 End-of-course learner meeting 31.13 Chart review 21.51 Written exam 51.57 Oral exam 50.79 Behavior ratings 41.25 OSCE31.55 5-point Likert 1=Never heard, 5=Use with almost all, >75%

13. Methods with Learners Exhibiting Difficult Median Proficiency Standard Deviation Identify learner having difficulty 41.07 Deliver negative feedback 21.39 Develop an action plan for correction of difficult 31.28 Re-evaluate the learner 3.50.74 Accurately evaluate the learner having difficulty 31.07 5-point Likert Scale, 1=None, 5=High

14. Results: Faculty Development Most useful faculty development topics  Teaching methodology  Evaluation techniques  Seminars on medical research

15. Format Preference for Faculty Development Median Standard Deviation PDA51.51 Online, web based 4.51.55 Live, group 41.09 Audio, CD 21.63 Video, DVD 21.46 5-point Likert Scale, 1=Never use, 5=Use often, prefer

16. Conclusions Faculty development strategies which address teaching and evaluation methods are desired and needed. Access to education resources is desired and needed. Time and communication are barriers to education even in the developing world. The self-reported level of use of interactive teaching techniques is high.

17. Conclusions Traditional methods are used to evaluate learners. While the proficiency in identifying learners exhibiting difficulty is high, proficiency with techniques to address the difficulty are low to moderate. PDAs, web-based and live group formats are preferred for faculty development programs.

18. Limitations Small sample size No response from NUR based faculty Self-assessment was a novel concept. Some terminology did not translate well. –Voluntary –Education concepts –Likert scale format Translation into French not preformed by a medical educator or researcher. Did not do two-way translation.

19. Implications The self-reported high use of interactive teaching, moderate level of proficiency with difficult learners and preference for PDA programs were unexpected and need to be explored. Needs are similar to the developed world. Faculty want to be effective regardless of country or continent. Our lessons learned can be used for more effective international needs assessment. This is a model for international faculty needs assessment.

20. Murakoze!

21. Results: Clinical Barriers to becoming the ideal clinical doctor  Heavy clinical load  Limited time  Lack of CME/CPD  Lack of proper laboratory services/equipment  Lack of referral systems/subspecialists  No support for career development Help you to become the ideal clinical doctor  CME/CPD/education resources  Study tours/exchanges at hospitals  Reduce the workload/Employ more doctors  Increase the number of collaborators