© Nuffield Trust Evaluation methods – where can predictive risk models help? Adam Steventon Nuffield Trust 8 July 2013.

Slides:

Advertisements

Similar presentations

When Should a Clinical Trial Design with Pre-Stratification be Used? Group 1.

Advertisements

Intro to Statistics Part2 Arier Lee University of Auckland.

Telehomecare: Outcomes and Patient Experiences

Clinical issues in telehealth: Unit M2 Dr Paul Rice David Barrett.

Presented by Malte Lierl (Yale University).  How do we measure program impact when random assignment is not possible ?  e.g. universal take-up  non-excludable.

Regression Discontinuity Design William Shadish University of California, Merced.

What Is Meant by "Real-World Data?"

National Research and Development Centre for Welfare and Health Knowledge for welfare and health / AL A propensity score approach to comparing.

© Nuffield Trust Inner North West London Integrated Care Pilot – year one evaluation 8 July 2013 Holly Holder Fellow in health policy Ian Blunt Senior.

© Nuffield Trust Annual Health Strategy Summit Dr Martin Bardsley Head of Research Nuffield Trust Twitter: #NTSummit.

The role of economic modelling – a brief introduction Francis Ruiz NICE International © NICE 2014.

Journal Club Alcohol, Other Drugs, and Health: Current Evidence September-October 2007.

Journal Club Alcohol and Health: Current Evidence November-December 2005.

Impact Evaluation: The case of Bogotá’s concession schools Felipe Barrera-Osorio World Bank 1 October 2010.

Journal Club Alcohol, Other Drugs, and Health: Current Evidence May–June 2009.

The Impact of Insurance Status on Hospital Treatment and Outcomes David Card, Carlos Dobkin and Nicole Maestas.

Cohort Studies Hanna E. Bloomfield, MD, MPH Professor of Medicine Associate Chief of Staff, Research Minneapolis VA Medical Center.

C-Reactive Protein: a Prognosis Factor for Septic Patients Systematic Review and Meta-analysis Introduction to Medicine – 1 st Semester Class 4, First.

DOES MEDICARE SAVE LIVES?

SPARRA Peter Martin Programme Principal Long Term Conditions/Joint Future Programmes.

© Nuffield Trust July 2010 Trends in Emergency Admissions in England

David Card, Carlos Dobkin, Nicole Maestas

Analysis of Virtual Wards: a multidisciplinary form of case management that integrates social and health care Martin Bardsley Director of Research Nuffield.

The Economic Impact of Intensive Case Management on Costly Uninsured Patients in Emergency Departments: An Evaluation of New Mexico’s Care One Program.

Advanced Statistics for Interventional Cardiologists.

Predictive Models for Health and Social Care: A Feasibility Study

© Nuffield Trust Predictive Risk 2012: Context Predictive R 13 June 2012 Martin Bardsley Head of Research Nuffield Trust.

Evaluating a whole systems approach to integrated care in North West London Nuffield Trust & London School of Economics 22 June 2015 Holly Holder

© Nuffield Trust June 22, 2012 Adam Steventon: Evaluating the Whole System Demonstrator trial Authors: Adam Steventon, Martin Bardsley Nuffield Trust.

Lecture 6 Objective 16. Describe the elements of design of observational studies: (current) cohort studies (longitudinal studies). Discuss the advantages.

© Nuffield Trust Review of aggregate assessment of providers of health and social care in England ('Rating Review‘)

© Nuffield Trust Improving hospital efficiency Authors: Jeremy Hurst and Sally Williams Selected figures January 2012.

PARR case finding tool Patients at risk of re- hospitalisation.

© Nuffield Trust November 2012 Evaluating the impact of the Marie Curie Nursing Service Xavier Chitnis, Theo Georghiou, Adam Steventon and Martin Bardsley.

Article Review Cara Carty 09-Mar-06. “Confounding by indication in non-experimental evaluation of vaccine effectiveness: the example of prevention of.

The Health Roundtable Connecting Care in the Community Presenter: Nicole McDonald, Manager Ongoing and Complex Care, CCLHD Central Coast LHD - NSW Innovation.

Evaluating the Efficacy of the Research Initiative for Scientific Enhancement (RISE) by using Propensity Scores to Identify a Matched Comparison Group.

© Nuffield Trust 22 June 2015 Matched Control Studies: Methods and case studies Cono Ariti

© Nuffield Trust 24 October 2015 NHS payment reform: evolving policy and emerging evidence Chief Economist: Anita Charlesworth.

Estimating Causal Effects from Large Data Sets Using Propensity Scores Hal V. Barron, MD TICR 5/06.

Lecture 7 Objective 18. Describe the elements of design of observational studies: case ‑ control studies (retrospective studies). Discuss the advantages.

Cost-effectiveness analysis of capitation vs. fee-for-service for Medicaid patients with severe mental illness Richard Grieve Visiting scholar UC Berkeley.

Presentations in this series 1.Introduction 2.Self-matching 3.Proxies 4.Intermediates 5.Instruments 6.Equipoise Avoiding Bias Due to Unmeasured Covariates.

Can Mental Health Services Reduce Juvenile Justice Involvement? Non-Experimental Evidence E. Michael Foster School of Public Health, University of North.

2nd Concertation Meeting Brussels, September 8, 2011 Reinhard Prior, Scientific Coordinator, HIM Evidence in telemedicine: a literature review.

© Nuffield Trust 4 July 2012 Future funding scenarios for the NHS and social care in England Adam Roberts Research Analyst

Getting the best of both worlds: exploiting observational data to maximise the relevance of RCTs for decision-making Health Economics at Kings seminar,

The Health Roundtable Postoperative IV Antibiotic Therapy for Children with Complicated Appendicitis: A Propensity Score-Matched Observational Study Presenter:

© Nuffield Trust Predictive risk and health care: an overview Research summary Selected figures March 2011.

Bangalore S, et al. β-Blocker use and clinical outcomes in stable outpatients with and without coronary artery disease. JAMA. 2012;308(13): ?

Randomized Assignment Difference-in-Differences

REBECCA M. RYAN, PH.D. GEORGETOWN UNIVERSITY ANNA D. JOHNSON, M.P.A. TEACHERS COLLEGE, COLUMBIA UNIVERSITY ANNUAL MEETING OF THE CHILD CARE POLICY RESEARCH.

Bilal Siddiqi Istanbul, May 12, 2015 Measuring Impact: Non-Experimental Methods.

© Nuffield Trust Social care and hospital use at the end of life Authors: Martin Bardsley and Theo Georghiou December 2010.

Gary L. Kamer Statistician OSB/DBS. 2 Statistical Issues at Time of PMA Review Clinical Study Design Excess All-cause Late Mortality (31 to 365 days)

The Impact of Insurance Status on Hospital Treatment and Outcomes David Card, Carlos Dobkin and Nicole Maestas.

Predicting risk of hospital admission and extracting GP data David Osborne Senior Public Health Information Analyst NHS Croydon.

© Nuffield Trust Uses of linked administrative data sets Adam Steventon, Nuffield Trust Royal Statistical Society 29 January 2013.

Propensity Score Matching in SPSS: How to turn an Audit into a RCT

Alexander Spermann University of Freiburg, SS 2008 Matching and DiD 1 Overview of non- experimental approaches: Matching and Difference in Difference Estimators.

The MOST project How have we changed the business model? 12 December 2012 Andrew McIntosh, Tunstall Healthcare Adam Steventon, Nuffield Trust.

CD4 trajectory among HIV positive patients receiving HAART in a large East African HIV care centre Agnes N. Kiragga 1, Beverly Musick 2 Ronald Bosch, Ann.

Matching methods for estimating causal effects Danilo Fusco Rome, October 15, 2012.

Addressing Confounding in Real-World Evidence Using Propensity Scores

Annals of Internal Medicine • Vol. 167 No. 12 • 19 December 2017

Risk adjustment using administrative and clinical data: model comparison

Improved Patient Outcomes Best Available Clinical Evidence Patient’s

Alcoholic liver disease in intensive care

JAMA Pediatrics Journal Club Slides: Factors Associated With Response Time to Physiologic Monitor Alarms Bonafide CP, Localio AR, Holmes JH, et al. Video.

Analysing RWE for HTA: Challenges, methods and critique

Presentation transcript:

© Nuffield Trust Evaluation methods – where can predictive risk models help? Adam Steventon Nuffield Trust 8 July 2013

© Nuffield Trust The problem with observational studies Eligible patients All patientsIntervention patients n54, % aged Prior emergency admissions Number chronic conditions Predictive risk score Intervention patients Source: Steventon et al (2012)

© Nuffield Trust Solutions, 1) before-after study

© Nuffield Trust Solutions, 2) regression adjustment Y = f(age, number of chronic conditions, prior emergency admissions, intervention status)

© Nuffield Trust Eligible patients Intervention patients Matched controls All patientsMatched controls Intervention patients n54, % aged Prior emergency admissions Number chronic conditions Predictive risk score Solutions, 3) Matched controls Source: Steventon et al (2012)

© Nuffield Trust How to select matched controls Propensity score (Rosenbaum and Rubin 1983) - Predictive risk of receiving the intervention Prognostic score (Hansen 2008) - Predictive risk of experiencing the outcome (e.g. emergency hospitalisation), in the absence of the intervention Genetic matching (Sekhon and Grieve 2012) - computer-intensive search algorithm

© Nuffield Trust Advantages / disadvantages Disadvantage – only allows for observed variables But Matching as ‘data pre-processing’ – reduces dependence of estimated intervention effects on regression model specification Intuitive? Good for routine monitoring – once controls found, data can be updated

© Nuffield Trust Overcoming regression to the mean using a control group Start of intervention

© Nuffield Trust Overcoming regression to the mean using a control group Start of intervention

© Nuffield Trust Overcoming regression to the mean using a control group Start of intervention

© Nuffield Trust Overcoming regression to the mean using a control group Start of intervention

© Nuffield Trust Solutions, 4) regression discontinuity Winning the next election Fraction of votes awarded to Democrats in the previous election Source: Lee and Lemieux (2009)

© Nuffield Trust What is being done at the moment? Telehealth studies in Pubmed, DescriptiveBefore after Dose response ControlledAll Number of studies Median number of patients in telemonitored group (range) 45 (40 to 851) 35 (7 to 17,025) (19 to 1,767)* 45 (7 to 17,025)* Endpoints Mortality2-136 Hospital use (or costs) Clinical (e.g. HbA1c) Patient reported outcomes (e.g. quality of life) Source: Steventon, Krief and Grieve (work in progress)

© Nuffield Trust References Lee DS, Lemieux T. Regression discontinuity designs in economics Available from: Sekhon JS, Grieve RD. A matching method for improving covariate balance in cost-effectiveness analyses. Health economics 2012;21:695–714. Rosenbaum P, Rubin D. The central role of the propensity score in observational studies for causal effects. Biometrika 1983;70:41–55. Hansen BB. The prognostic analogue of the propensity score. Biometrika 2008;95:481–8. Steventon A, Bardsley M, Billings J, Georghiou T, Lewis GH. The role of matched controls in building an evidence base for hospital- avoidance schemes: a retrospective evaluation. Health services research 2012;47:1679–98.

© Nuffield Trust 8 July Sign-up for our newsletter: Follow us on Twitter: © Nuffield Trust