Development of a course on microsimulation of health Philippe Finès February 2011

2 Statistics Canada Statistique Canada Update on the webinars / consultations on POHEM  Basics in POHEM programming  How to implement a disease  How to implement risk factors  Table building  BioBrowser  Debugging  Utilisation of POHEM-OA  Interpretation of parameters  Computation of variance  Scenario building  Calibration, sensitivity analysis  Calibration  Sensitivity analysis  Uncertainty  Critical appraisal  CHD  Salt  CISNET models  HTA  101 courses  CVD  Inequalities  Development of POHEM-Heart disease  COPD  Others  STAR Online resources (Wiki, repository, etc)

3 Statistics Canada Statistique Canada Update on the webinars / consultations on POHEM  Conclusions: Despite organizational constraints (simultaneous availability of all, technical aspects of webinars, etc…) we were able to provide interventions in a relatively short time The contents of the interventions was by nature interactive, i.e. they addressed the specific needs of the trainees From this experience, we are able to determine what would be the ideal course  (Feedback from our July 2010 meeting) It was generally agreed that a course should be concept-based with a supplemental course on programming offered later. The idea of doing a 6-8 week course on the critical appraisal of models was suggested and agreed on. The first step towards this will involve the team performing a critical appraisal and then creating a reading course. Another idea was to teach the concepts of different models and for a final project to have students describe a model as a solution to an issue but not necessarily build it. The Modeling division at StatsCan has a “baby Modgen” that can be used to teach simulation methods. There is also the possibility of simplifying a current version of POHEM for students to run a simple simulation. Action Items: Team to perform critical appraisal of models

4 Statistics Canada Statistique Canada Principles used in the course  Simulation is used in lieu of clinical trials or surveys or censuses.. when and why? i.e. microsimulation is to be used in conjunction with other data analysis methods i.e. microsimulation may be used to predict or to explain  Simulation is not “let’s put together data and see what happens” i.e. microsimulation must be run with the same rigor than any other data analysis method  Microsimulation is complex! hard to reproduce individual trends hard to reproduce joint distributions of covariates balance between reproducing past and projecting future problem with variables missing in the model computation of variance sensitivity analysis

5 Statistics Canada Statistique Canada  This course is intended to address needs and gaps among epidemiology and medicine audience be a fine balance between  theory and practice  general and particular  data-oriented and policy-oriented  This course is addressed to graduate students who already have a background on statistics and epidemiology.  It consists of 15 3-hour sessions and gives 3 credits.  Evaluation will be done from attendance in class, small tests on concept acquisition and the personal work described in (4) below. Context of the course

6 Statistics Canada Statistique Canada  At the end of the course, the student should be able to know the essentials of simulation in the context of health (microsimulation, agent-based modelling), including:  pathway of disease  health services  economics/decision analysis know the concepts and steps involved in simulation in the context of health, including:  literature review  justification of simulation modeling  development of model  testing, validation, calibration Objectives of the course

7 Statistics Canada Statistique Canada Objectives – details 1.know the essentials of simulation why simulate? micro/macro simulation discrete/continuous time levels of complexity use of probability in simulation strengths/weaknesses of simulation (comparison with other tools) basics on programming in microsimulation 2.know the concepts involved in simulation in the health context how to model a disease, the interventions, the screening how to model relative risk, specificity/sensitivity, hazard, … the need to validate the role of data (as parameters, as output, as validation tool) some typical models (diseases) used in literature

8 Statistics Canada Statistique Canada 1. Essentials of simulation in the context of health {1 session} 2.Concepts involved in simulation in the context of health {4 sessions} 3.Simulation as a scientific process {7 sessions} 4.Personal work {3 sessions}. Outline

9 Statistics Canada Statistique Canada 1. Essentials of simulation in the context of health {1 session} reasons to simulate strengths/weaknesses of simulation (comparison with other tools) types of simulation (macro/micro/agent-based, cell-based, discrete/continuous time) scope of simulation (health, health care, environmental modeling) levels of complexity in simulation models use of probability in simulation Outline Description of principles of simulation, simulation as a scientific process. Choice of simulation projects.

10 Statistics Canada Statistique Canada 2.Concepts involved in simulation in the context of health {4 sessions} Reminder of basic concepts (illustrated for some typical diseases) {1}  Etiology of disease (Incidence, prevalence, risk factors, relative risks, survival, …)  Test and Screening (Sensitivity, specificity, predictive value of a test, …)  Treatment (Efficiency, efficacy, Cost-effectiveness[1], …)[1] Basics on Health care and decision analysis {1} Modeling of these concepts {2} Outline Students can get accustomed on how to create simple simulation models with Excel, R, SAS, etc. Complex languages such as POHEM should be reserved only to produce examples

11 Statistics Canada Statistique Canada 3.Simulation as a scientific process {7 sessions} Research question, literature review and justification for simulation {1} Data analysis {1}  Sources of data  Published and ad hoc analyses Development and testing {2}  Development of model  Production of results and testing Challenges of simulation {3}  Calibration  Sensitivity analysis[2][2]  Principles of validation[3][3]  Critical appraisal of some published models of simulation[4][4] Outline These concepts are general enough to be applied to most models of simulation. Students may apply them to develop their own models (ex: queueing theory for hospital simulations) and use either general software (e.g. Excel) or specific softwares easy to use (some of them are free)

12 Statistics Canada Statistique Canada 4.Personal work {3 sessions}. In this part, the student is invited to work more thoroughly on one of the concepts seen in the course: programming of a model critical appraisal and comparative review of published models development of a measure of quality of model development of a measure of cost-effectiveness etc Outline Students should be encouraged to be proactive in the choice of the model they want to develop, as long as it fits the objectives of the course. One of the sessions could be at the beginning (students present their draft and their plan); one of them could be at the end (presentations in class)

13 Statistics Canada Statistique Canada  [1] Jennifer J. Telford, Adrian R. Levy, Jennifer C. Sambrook, et al. The cost-effectiveness of screening for colorectal cancer, CMAJ DOI: /cmaj (among others) [1]  [2] AH Briggs, AM Gray. Handling uncertainty when performing economic evaluation of healthcare interventions. Health Technology Assessment 1999; Vol. 3, No. 2 (among others) [2]  [3] Jacek A Kopec, Philippe Finès, Douglas G Manuel, et al., Validation of population-based disease simulation models: a review of concepts and methods, BMC Public Health 2010, 10:710 (among others) [3]  [4] Harindra C. Wijeysundera; Márcio Machado; Farah Farahati; et al., Association of Temporal Trends in Risk Factors and Treatment Uptake With Coronary Heart Disease Mortality, , JAMA. 2010;303(18): (doi: /jama ) (among others) [4] Notes

14 Statistics Canada Statistique Canada Next steps  Outline of the course Evaluation of the 1 st draft (dry run, focus group, pilot students) Modification and finalisation of the outline  Administrative aspects Ad hoc course at UBC

15 Statistics Canada Statistique Canada References        Zaidi, A. and K. Rake (2001), Dynamic Microsimulation Models: A Review and Some Lessons for SAGE, SAGE Discussion Paper 02, London School of Economics  Alain Bélanger. DMO 6321 : Perspectives et micro-simulations, Plan du cours, Automne 2009, Institut national de la recherche scientifique, Urbanisation Culture Société