Longitudinal Methods for Pharmaceutical Policy Evaluation Dennis Ross-Degnan, ScD Harvard Medical School and Harvard Pilgrim Health Care WHO Collaborating Center in Pharmaceutical Policy Boston, USA

Session Objectives  Touch on key methodological issues in longitudinal studies to evaluate: Pharmaceutical policy changes Planned interventions  Hear experiences of researchers who have used longitudinal data in a range of settings  Introduce commonly-used statistical methods Interrupted time series and survival analysis  Discuss Other experiences and perspectives Best practices and areas for methods development

Using Routine Data for Pharmaceutical Policy Research  Pharmacy procurement and sales Public, mission, private sector Centralized, supply chain, institutional Volume, cost  Clinical care and pharmacy dispensing Inpatient, outpatient, retail pharmacy Electronic records Manual systems  Insurance reimbursement Claims, adjudicated payments Critical Issues Completeness Consistency Coding

Common Methodological Issues in Longitudinal Policy Evaluations  Time  Study design  Sample selection  Data quality  Data organization  Statistical approach

Issues Related to Time  Key analytic variable for longitudinal research Errors common: recording, coding Importance of definitions (e.g., medication gaps)  Defining policy change point Single point in time, instantaneous effects Implementation spread over time Co-interventions  Dynamics of policy impacts Anticipatory changes, lagged response Non-linear changes  Study period and unit of aggregation Depends on data source and sample size Optimal number of data points per policy period?

Issues in Study Design  Appropriate study units Whose behavior will change? External policy influences  Timing of implementation (prospective) Opportunity for randomization? Staggered implementation?  Comparisons and contrasts Challenge of identifying similar groups or behaviors unaffected by intervention Intended and unintended effects High vs. low risk

Issues in Sample Selection  Facilities, prescribers, patients Optimal sample structure? Importance of denominators, continuity Defining prevalent and incident diagnoses  Medications Trade-offs among therapeutic alternatives All vs. selected categories  How many is enough? Representativeness Need for precision Problem of clustering

Issues in Data Quality  Many challenges in using routine data Usually not collected for research Changes in data systems or routines  Common data quality issues Combining data across facilities Missingness Unusual patterns, wild data points  Importance of diagnostics Graphical display Evaluating patterns of variability, missingness Comparing baseline patterns in subgroups

Issues in Data Organization  Choice of level of analysis Aggregated across all units Separately by logical units (facility, prescriber) Patient-level analysis  Patient subgroups Continuing vs. new patients Clinical risk subgroups  Medication data Therapeutic classification and organization Policy-induced switching (market share analysis)

Issues in Statistical Approach  Study design, sampling, and statistical approach must go hand in hand Duration of available data is key factor Level of analysis  Validity in longitudinal policy change models Baseline serves as counterfactual Co-intervention is the major confounder Need to understand context and stability of system

Presenters  Christine Lu, USA Market utilization or sales data (Abstract 878)  Sauwakon Ratanawijitrasin, Thailand Electronic clinical and pharmacy data (Abstract 811)  Ricardo Perez-Cuevas, Mexico Electronic medical record data (Abstract 1118)  Joshua Kayiwa, Uganda Routine data from manual systems (Abstract 505)  Mike Law, Canada Overview of common analytic approaches

Listen, participate, enjoy… Thank you!