2. Methods for evaluation of a prescribing curriculum for Australian senior medical students AJ Smith, SR Hill, T Tasioulas, NL Cockayne.

3. Objectives  To design, field-test, implement and evaluate the impact of a modular, problem-based web-interactive prescribing curriculum based on the WHO “Guide to Good Prescribing”.

4. Summary of methods  Curriculum developed by a consortium from all medical schools.  Made web-based, interactive by “Knowledge South” and the University of South Australia.  Field-tested for content and ease of navigation by medical students and staff.  Made available to medical schools - could not be mandated.

5. Evaluative methods – the ideal  A pre-planned randomised allocation of medical schools to receive/not receive the curriculum with pre- and post-testing of each group on ability to select medicines discussed in the modules.  OR  Randomisation within each school with delayed introduction of the curriculum to one of the two student groups – but potential problem with “contamination”

6. Obstacles to the ideal.  No advance knowledge of which, if any, school would use the curriculum  No knowledge of whether all or only part of the curriculum would be used  No ability to do delayed introduction as students needed to be moving through at the same rate if the curriculum replaced conventional teaching, which was what teachers wanted.

7. What was feasible ?  “Hits” on the website organised by school  In-depth interviews with staff, on-line feedback from 107 med. students representing the schools that used the curriculum  Before-and-after testing of a cohort of students from one school.

8. What were the problems in interpretation of the results ?  “Hits” only mean that the page has been opened – not an assessment of learning.  Staff respondents were uniformly supportive  Student responses may have only come from the enthusiasts and not be fully representative.  Before-after results show that compulsion works but are they a fair test of the program’s ability to change prescribing ability ? Were there any other influences working at the same time? No control group.

9. How would we do it differently ?  Difficult unless we can construct control groups and, perhaps, conduct the testing over a shorter period.  If schools do not take up the curriculum, seek to use their students as controls for further before-after tests.

10. A PRESCRIBING CURRICULUM FOR AUSTRALIAN MEDICAL STUDENTS Smith AJ, Hill SR,Tasioulas T,Cockayne NL. National Prescribing Service, Sydney & Clinical Pharmacology Unit, University of Newcastle, Australia

11. Background  The Australian National Prescribing Service (NPS) not-for-profit, independent organisation  Activities aim to improve the overall quality of use of medicines in Australia  Development of curriculum as a response to:  Documented sub-optimal prescribing by doctors  A perceived deficit in prescribing education for senior students in medical schools - despite good programs in pharmacology/clinical pharmacology

12. Objectives  To design, field-test, implement and evaluate the impact of a modular, problem- based web-interactive prescribing curriculum based on the WHO “Guide to Good Prescribing”

13. Methods (1) Design  Jointly sponsored by NPS and the Australasian Society for Clinical and Experimental Pharmacologists and Toxicologists (ASCEPT)  Collaboration with teachers from each of the Australian medical schools  Clinical conditions were selected from those that interns claimed were common and raised prescribing issues  The 12 written cases were converted to web-based interactive form by Knowledge South and the University of South Australia

14. Methods (2) Field-testing and implementation  The 12 modules were field-tested for ease of navigation and for content by medical students and staff  The curriculum could not be mandated and uptake was dependent on NPS advocacy and the influence of the teachers who had helped construct the program  The completed modules were made available to each of the 11 Australian medical schools in January, 2002

15. Methods (3) Evaluation  Measurement of ‘hits’ on the website by each medical school  Structured interviews with 15 teachers from 9 medical schools and online survey of 121 medical students from 5 medical schools  An assessment of students’ ability to select drugs for four common conditions:  before using the curriculum  after one year’s voluntary use  after a further one year’s compulsory use

16. Results (1) *Medical Schools have been de-identified

17. Results (2) Qualitative feedback: In-depth interviews with teaching staff  All used it as a tool for individual self-directed learning tool  five with added case-discussions  Five schools made the curriculum assessable in final examinations  Issues/problems:  Technical aspects e.g. navigation and browser capability  “Need to engage more teaching staff within faculty”  Generally, “an excellent, practical resource”

18. Results (3) Qualitative feedback: student online survey  97/107 (91%) therapeutic content appropriate to needs  92/107 (86%) felt better able to prescribe after using the curriculum  57 (54%) had developed their own personal formulary – a major goal of the program

19. Results (4) Improvements in selecting appropriate drugs in each condition were significant (  2 =37, p=0.01)

20. Conclusions  The program has proved very popular with students and staff  Uptake in the first two years has been better than predicted but the program needs more ‘marketing’  The modules have now been revised and plans are in place for more thorough evaluation in 2004  Website URL: http://nps.unisa.edu.au