External Validity of Trials. Background External or ecological validity refers to whether the results of the trial can be generalised to the general clinical.

Slides:

Advertisements

Similar presentations

Workshop C – Evaluation Rod Taylor Complex Interventions Research Framework Masterclass 2010.

Advertisements

Nina Dunham R&D Manager

March 2004 University of Bristol Leverhulme Centre for Market and Public Organisation Comments on Trying it Out: The Role of Pilots in Policy Making Carol.

9 February 2012 NIHR-funded Surgical Research: NIHR RfPB Award Holder Matt Costa.

The Bahrain Branch of the UK Cochrane Centre In Collaboration with Reyada Training & Management Consultancy, Dubai-UAE Cochrane Collaboration and Systematic.

Patient Preference and Comprehensive Cohort Designs.

Unequal Randomisation

Use of Placebos in Controlled Trials. Background The traditional ‘double-blind’ RCT uses a placebo to conceal allocation. There are a number of advantages.

NIHR Research Design Service London Enabling Better Research Forming a research team Victoria Cornelius, PhD Senior Lecturer in Medical Statistics Deputy.

ATLAS Steering Committee: 24 September 2005 Steering Committee meeting, 24 th September 2005 University of Oxford Examination Schools.

KINE 4565: The epidemiology of injury prevention Randomized controlled trials.

1 Health and Disease in Populations 2002 Week 9 – 2/5/02 Randomised controlled trials 2 Dr Jenny Kurinczuk.

‘Cohort multiple RCT’ design workshop Clare Relton & Jon Nicholl School of Health and Related Research (ScHARR) Faculty of Medicine University of Sheffield.

Improving the Economic Efficiency of Clinical Trials.

Clinical Trials Importance in future therapies. What are the Requirements to Produce New Drugs? Drug must work significantly better than a control treatment.

Recruitment to Trials. Background Recruitment of participants is a VERY important issue. The general consensus is that most trials under recuit.

Guidelines and Guideline Development HINF Medical Methodologies Session 13.

Pragmatic or explanatory trial? Hywel Williams University of Nottingham with help from Daniel Bratton and Andrew Nunn MRC Clinical Trials Unit HTA reference.

Journal Club Alcohol, Other Drugs, and Health: Current Evidence May-June 2007.

N = 1, Cross-Over Trials and Balanced Designs. N = 1 Trials Trials can be undertaken with just one participant. If the condition is a chronic relapsing.

Pragmatic Randomised Trials. Background Many clinical trials take place in artificial conditions that do not represent NORMAL clinical practice. Often.

The Scope of Musculoskeletal Disease Treatment and Costs Prof Stephen Graves University of Melbourne.

Research Priorities for Osteoarthritis of the Knee Chard JA, Tallon D, Dieppe PA.

As noted by Gary H. Lyman (JCO, 2012) “CER is an important framework for systematically identifying and summarizing the totality of evidence on the effectiveness,

Design of Clinical Trials of Antibiotic Therapy for Acute Otitis Media

Are the results valid? Was the validity of the included studies appraised?

Inference in practice BPS chapter 16 © 2006 W.H. Freeman and Company.

Intervention Studies Principles of Epidemiology Lecture 10 Dona Schneider, PhD, MPH, FACE.

East & South East England Specialist Pharmacy Services East of England, London, South Central & South East Coast Research methods: answering questions.

Copyright © 2010 Pearson Education, Inc. Slide

EVIDENCE BASED MEDICINE Health economics Ross Lawrenson.

Healthcare Philosophies. What are they? As with many things in life there are a few different approaches to medical knowledge. Think of it like different.

Criteria for Assessing The Feasibility of RCTs. RCTs in Social Science: York September 2006 Today’s Headlines: “Drugs education is not working” “ having.

EVIDENCE BASED MEDICINE Effectiveness of therapy Ross Lawrenson.

LT 4.2 Designing Experiments Thanks to James Jaszczak, American Nicaraguan School.

Understanding real research 4. Randomised controlled trials.

Basic Economic Analysis David Epstein, Centre for Health Economics, York.

1 SCREENING. 2 Why screen? Who wants to screen? n Doctors n Labs n Hospitals n Drug companies n Public n Who doesn’t ?

Plymouth Health Community NICE Guidance Implementation Group Workshop Two: Debriding agents and specialist wound care clinics. Pressure ulcer risk assessment.

Urgent and planned orthopaedic surgery at Imperial College Healthcare NHS Trust.

February February 2008 Evidence Based Medicine –Evidence Based Medicine Centre –Best Practice –BMJ Clinical Evidence –BMJ Best.

What is a non-inferiority trial, and what particular challenges do such trials present? Andrew Nunn MRC Clinical Trials Unit 20th February 2012.

1 Health and Disease in Populations 2002 Session 8 – 21/03/02 Randomised controlled trials 1 Dr Jenny Kurinczuk.

Compliance Original Study Design Randomised Surgical care Medical care.

Can patients be too mild, too severe or too old for thrombolysis? Professor Peter Sandercock University of Edinburgh ESC Hamburg 27 th May 2011 Disclosures.

Course: Research in Biomedicine and Health III Seminar 5: Critical assessment of evidence.

HTA Efficient Study Designs Peter Davidson Head of HTA at NETSCC.

Colorectal Cancer Screening Implementation of a public health programme An Expert Group on Colorectal Cancer Screening Cancer Society of Finland, Finnish.

Who is involved in making NICE guidance recommendations and what evidence do they look at? Jane Cowl, Senior Public Involvement Adviser Tommy Wilkinson,

Falls and Fall Prevention. Prevalence of Falls in Older Adults  33% of older adults fall each year  Falls are the leading cause of fatal and nonfatal.

Critical Appraisal Course for Emergency Medicine Trainees Module 3 Evaluation of a therapy.

MONDAY 01/02/2016 Professional English in Use, Medicine Medical Practitioners 1.

‘Enhancing musculoskeletal research in primary care: engaging a community musculoskeletal physiotherapy service in research studies’ Presented by : Carol.

An Introduction to Clinical Trials and Pharmaceutical Statistics Workshop Robbie Peck University of Bath Student-Led Symposia 16 th Feb 2016.

PRAGMATIC Study Designs: Elderly Cancer Trials

How Chiropractors Treat Back Pain ?. Back pain most often affects people between the ages of 30 and 60. It might be difficult to pinpoint the cause of.

Trials Adrian Boyle.

New Advance in Back Pain Treatment: Stem Cell Therapy

Research Questions Does integration of behavioral health and primary care services, compared to simple co-location, improve patient-centered outcomes in.

Confidence Intervals and p-values

Within Trial Decisions: Unblinding and Termination

Randomized Trials: A Brief Overview

THE EFFECTIVENESS OF ANNUAL ZOLEDRONIC ACID INFUSION VERSUS ORAL BISPHOSPHONATE: A MODELLING APPROACH Terence Ong1, 2, Matthey Jones3, Opinder Sahota1.

2017 National Patient Safety Goals

Challenges of statistical analysis in surgical trials

Matthew H. Liang, MD Malcolm P. Rogers, MD Ira R. Katz, MD, PhD

How to apply successfully to the NIHR HTA Board?

Analysing RWE for HTA: Challenges, methods and critique

Acupuncture for Chronic Pain

Presentation transcript:

External Validity of Trials

Background External or ecological validity refers to whether the results of the trial can be generalised to the general clinical population. Randomisation per se does NOT make a study externally valid.

Validity Issues Are the ‘correct’ patients being included in the trials? Are the ‘correct’ practitioners taking part in the study? Are the ‘correct’ facilities being used?

Trial Problems Many if not most trials are undertaken in circumstances that are far removed from ‘usual’ clinical practice. Usually clinicians running trials are content experts yet the results are to be applied by non-experts.

Osteoporosis Trials Most of the large osteoporosis trials with fracture as an outcome recruited participants from ‘bone’ clinics. Patients under care of clinical experts but results are expected to be applied by clinicians who are not experts.

Participants Usually participants who go into trials are different from those who do not. These ‘storm troopers’ of patients usually comply with instructions better than average. Combined with different clinicians and patients results can be different from the ‘real’ world.

Trial participants v non % (n) withTook part in trial n = Did not take part in trial, n = risk factor42.4 (597 ) 62.9 (1 141) risk factors40.9 (576)29.4 (533) risk factors13 (183)6.9 (126) risk factors3.8 (53)0.8 (14).000

Crude risk of fracture Type of fracture Took part in trial n = Did not take part in trial, n = P value Overall6.5%10.4%.000 Hip1.4%1.8%.405 Wrist2.5%3.8%.044

Risk of fracture (adjusted) Type of fracture OR (adjusted) 95% CI lower upper P value Overall Hip Wrist

Alendronate Alendronate is a bisphosphonate for osteoporosis treatment. Large trials conducted in expert clinical centres showed it reduced fractures with few or no side-effects. Things were different when it was licensed and ‘real’ clinicians used the drug.

Alendronate problems To take a bisphosphonate one needs to take it on an empty stomach with a large glass of water and remain upright for at least 2 hours. Otherwise the drug can get affect the oesophagus and cause ulceration. In real life this is what happened and led to a warning. Trial data gave to evidence this was a problem.

The Wrong Solution Often once the phase III trial has been finished companies finance a ‘real’ world non-randomised study. This is usually nothing more than a marketing exercise and produces worthless data on ‘real’ world experience of the problem.

A correct solution Involves some form of randomisation to eliminate confounding. For example one might undertake a cluster trial of giving the drug non- specialist physicians to use of their patients BEFORE the drug is widely available. This will produce more robust data.

Alternatively We should try and design our trials to be as pragmatic as possible so that the results are widely generalisable. This means recruiting non-expert clinicians and their patients. Using a pragmatic design rather than an explanatory design.

Back Pain Trials An early RCT of chiropractic manipulation for back pain was criticised by physiotherapists that the chiropractic was undertaken in private practice. May have been a ‘Bach’ effect.

MRC back pain trial To address the issue of nice premises the MRC back pain trial included an arm where patients were randomised to be treated on private premises or NHS premises. Increased the costs of the trial hugely.

Surgical Trials Often surgical trials have poor external validity because the new technique is usually developed in a teaching hospital among ‘expert’ surgeons (e.g. laparoscopic methods). It is unlikely the results of surgery undertaken in ‘top’ hospitals would be the same as a ‘bog-standard’ DGH.

Big and Simple Try and make the trial large, as it allows for various PRE-SPECIFIED sub-group analyses (specialists vs generalists). Simple which means one can get as many in as possible.

Summary Most trials will NEVER be able to recruit exactly the same types of participants as those who receive treatment in clinical practice. PRAGMATIC trials have the strongest external validity and usually one of these should be done before implementation.