2. An Evaluation of the WHO Rational Pharmacotherapy Teaching Program Hill S, Smith AJ, Thambiran M, Walkom E, Hogerzeil HV

3. Background  Guide to Good Prescribing developed in early 1990s  Teaching pharmacotherapy:  6 step process  Problem based learning  Assessment  RCT (1995) demonstrating effect on prescribing skills  Training program developed

4. Study outline  11 WHO –GGP train the trainer courses 1994-2003  Impact? Evaluation?  Follow-up of all participants attempted  Self report of impact, use and suggestions about course  Control group of institutions without participants, matched for type of institution and country

5. Results summary  So far….  Response rate 36/247, 12 controls  Anecdotes: change in teaching methods, change in other institutions, ‘spillover’ into other groups  Control group: used or know of GGP – 75%

6. Planning the evaluation? Resources?  No master plan at the time of the courses…..many years later….  Post-hoc grant from WHO  Where are the participants?  Identifying a comparison/control group?  Measure of outcome?  No objective ‘pre’ measurement

7. Design  Observational, cross-sectional questionnaire  Comparison group to assess secular trends  Pragmatic choice based on time since intervention  Repeated measures?

8. Strengths and weaknesses Strengths  Some data better than none  Some attempt at comparison  Established database for future measurements Weaknesses  Post hoc (a long way!)  Response rate  Comparison group  Self report  Lack of gold standard outcome measure: Change in prescribing

9. Lessons learned  Plan at the beginning  Define an outcome  Establish contacts early and keep them  Establish a comparison group – probably can’t randomise  …. Or you could….

10. Gold standard  Planning!!!!  What is the question?  RCT of training program:  Randomise by institution, within country  Baseline objective measure of type of teaching program  Surrogate outcome is change in teaching program to PBL with GGP principles  Final outcome (later follow-up of cohort of students): impact on prescribing

11. Is it cost effective?  Cost of a course ?  conservatively $US100000  247 trained, total cost $US1million  Outcome:  Sustained curricular change  Average cost/person reporting sustained curricular change  $6250 - $40000  Is this value for money?