1 Meeting the Future in Managing Chronic Disorders: Individually Tailored Strategies S.A. Murphy Herbert E. Robbins Collegiate Professorship in Statistics.

Slides:



Advertisements
Similar presentations
13 Principles of Effective Addictions Treatment
Advertisements

Piloting and Sizing Sequential Multiple Assignment Randomized Trials in Dynamic Treatment Regime Development 2012 Atlantic Causal Inference Conference.
Treatment Effect Heterogeneity & Dynamic Treatment Regime Development S.A. Murphy.
11 Confidence Intervals, Q-Learning and Dynamic Treatment Regimes S.A. Murphy Time for Causality – Bristol April, 2012 TexPoint fonts used in EMF. Read.
Experimental Trials. Oslin ExTENd Late Trigger for Nonresponse 8 wks Response TDM + Naltrexone CBI Random assignment: CBI +Naltrexone Nonresponse Early.
1 Meeting the Future in Managing Chronic Disorders: Individually Tailored Strategies S.A. Murphy Univ. of Michigan Oberlin College, Feb. 20, 2006.
Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making S. Murphy & J. Pineau American Control Conference Special Session.
Experimenting to Improve Clinical Practice S.A. Murphy AAAS, 02/15/13 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.:
1 Dynamic Treatment Regimes Advances and Open Problems S.A. Murphy ICSPRAR-2008.
Causal Inference and Alternative Explanations S.A. Murphy Univ. of Michigan May, 2004.
1 Developing Adaptive Treatment Strategies using MOST Experimental Designs S.A. Murphy Univ. of Michigan Dallas: December, 2005.
Methodology for Adaptive Treatment Strategies for Chronic Disorders: Focus on Pain S.A. Murphy NIH Pain Consortium 5 th Annual Symposium on Advances in.
Experiments and Dynamic Treatment Regimes S.A. Murphy Univ. of Michigan JSM: August, 2005.
SMART Designs for Constructing Adaptive Treatment Strategies S.A. Murphy 15th Annual Duke Nicotine Research Conference September, 2009.
Dynamic Treatment Regimes, STAR*D & Voting D. Lizotte, E. Laber & S. Murphy LSU ---- Geaux Tigers! April 2009.
Substance Abuse, Multi-Stage Decisions, Generalization Error How are they connected?! S.A. Murphy Univ. of Michigan CMU, Nov., 2004.
An Experimental Paradigm for Developing Dynamic Treatment Regimes S.A. Murphy Univ. of Michigan March, 2004.
Constructing Dynamic Treatment Regimes & STAR*D S.A. Murphy ICSA June 2008.
1 Dynamic Treatment Regimens S.A. Murphy PolMeth XXV July 10, 2008.
SMART Designs for Developing Adaptive Treatment Strategies S.A. Murphy K. Lynch, J. McKay, D. Oslin & T.Ten Have CPDD June, 2005.
Dynamic Treatment Regimes: Challenges in Data Analysis S.A. Murphy Survey Research Center January, 2009.
Experiments and Dynamic Treatment Regimes S.A. Murphy Univ. of Michigan Florida: January, 2006.
SMART Experimental Designs for Developing Adaptive Treatment Strategies S.A. Murphy NIDA DESPR February, 2007.
Hypothesis Testing and Dynamic Treatment Regimes S.A. Murphy Schering-Plough Workshop May 2007 TexPoint fonts used in EMF. Read the TexPoint manual before.
An Experimental Paradigm for Developing Adaptive Treatment Strategies S.A. Murphy Univ. of Michigan UNC: November, 2003.
Experiments and Dynamic Treatment Regimes S.A. Murphy Univ. of Michigan PSU, October, 2005 In Honor of Clifford C. Clogg.
Planning Survival Analysis Studies of Dynamic Treatment Regimes Z. Li & S.A. Murphy UNC October, 2009.
Statistical Issues in Developing Adaptive Treatment Strategies for Chronic Disorders S.A. Murphy Univ. of Michigan CDC/ATSDR: March, 2005.
SMART Experimental Designs for Developing Adaptive Treatment Strategies S.A. Murphy RWJ Clinical Scholars Program, UMich April, 2007.
Hypothesis Testing and Dynamic Treatment Regimes S.A. Murphy, L. Gunter & B. Chakraborty ENAR March 2007.
1 SMART Designs for Developing Adaptive Treatment Strategies S.A. Murphy K. Lynch, J. McKay, D. Oslin & T.Ten Have UMichSpline February, 2006.
A Finite Sample Upper Bound on the Generalization Error for Q-Learning S.A. Murphy Univ. of Michigan CALD: February, 2005.
Methodology for Adaptive Treatment Strategies R21 DA S.A. Murphy For MCATS Oct. 8, 2009.
An Experimental Paradigm for Developing Adaptive Treatment Strategies S.A. Murphy Univ. of Michigan ACSIR, July, 2003.
Dynamic Treatment Regimes, STAR*D & Voting D. Lizotte, E. Laber & S. Murphy Psychiatric Biostatistics Symposium May 2009.
Journal Club Alcohol, Other Drugs, and Health: Current Evidence January–February 2011.
An Experimental Paradigm for Developing Adaptive Treatment Strategies S.A. Murphy Univ. of Michigan February, 2004.
Experiments and Dynamic Treatment Regimes S.A. Murphy Univ. of Michigan Yale: November, 2005.
1 Possible Roles for Reinforcement Learning in Clinical Research S.A. Murphy November 14, 2007.
Experiments and Dynamic Treatment Regimes S.A. Murphy Univ. of Michigan April, 2006.
Exploratory Analyses Aimed at Generating Proposals for Individualizing and Adapting Treatment S.A. Murphy BPRU, Hopkins September 22, 2009.
SMART Experimental Designs for Developing Adaptive Treatment Strategies S.A. Murphy ISCTM, 2007.
Experiments and Adaptive Treatment Strategies S.A. Murphy Univ. of Michigan Chicago: May, 2005.
Susan Murphy, PI University of Michigan Acknowledgements: MCAT network and NIH The Goal To facilitate methodological collaborations necessary for producing.
1 Dynamic Treatment Regimes: Interventions for Chronic Conditions (such as Poverty or Criminality?) S.A. Murphy Univ. of Michigan In Honor of Clifford.
SMART Designs for Developing Dynamic Treatment Regimes S.A. Murphy Symposium on Causal Inference Johns Hopkins, January, 2006.
Experiments and Dynamic Treatment Regimes S.A. Murphy At NIAID, BRB December, 2007.
1 Machine/Reinforcement Learning in Clinical Research S.A. Murphy May 19, 2008.
Adaptive Treatment Strategies S.A. Murphy CCNIA Proposal Meeting 2008.
Adaptive Treatment Strategies S.A. Murphy Workshop on Adaptive Treatment Strategies Convergence, 2008.
Practical Application of Adaptive Treatment Strategies in Trial Design and Analysis S.A. Murphy Center for Clinical Trials Network Classroom Series April.
Experiments and Dynamic Treatment Regimes S.A. Murphy Univ. of Michigan January, 2006.
1 Variable Selection for Tailoring Treatment S.A. Murphy, L. Gunter & J. Zhu May 29, 2008.
Hypothesis Testing and Adaptive Treatment Strategies S.A. Murphy SCT May 2007.
Adaptive Treatment Design and Analysis S.A. Murphy TRC, UPenn April, 2007.
Adaptive Treatment Strategies: Challenges in Data Analysis S.A. Murphy NY State Psychiatric Institute February, 2009.
1 Meeting the Future in Managing Chronic Disorders: Individually Tailored Strategies S.A. Murphy Univ. of Michigan In Honor of Clifford C. Clogg.
Sequential, Multiple Assignment, Randomized Trials and Treatment Policies S.A. Murphy UAlberta, 09/28/12 TexPoint fonts used in EMF. Read the TexPoint.
Overview of Adaptive Treatment Regimes Sachiko Miyahara Dr. Abdus Wahed.
SMART Case Studies Module 3—Day 1 Getting SMART About Developing Individualized Adaptive Health Interventions Methods Work, Chicago, Illinois, June
Sequential, Multiple Assignment, Randomized Trials and Treatment Policies S.A. Murphy MUCMD, 08/10/12 TexPoint fonts used in EMF. Read the TexPoint manual.
Sequential, Multiple Assignment, Randomized Trials Module 2—Day 1 Getting SMART About Developing Individualized Adaptive Health Interventions Methods Work,
RAMAR  SINCE 1980, RAMAR HAS BEEN A VITAL PART OF RECOVERY FOR CHRONICALLY ADDICTED RECOVERY FOR CHRONICALLY ADDICTED INDIVIDUALS IN NEED IN SUMMIT COUNTY.
1 SMART Designs for Developing Adaptive Treatment Strategies S.A. Murphy K. Lynch, J. McKay, D. Oslin & T.Ten Have NDRI April, 2006.
Motivation Using SMART research designs to improve individualized treatments Alena Scott 1, Janet Levy 3, and Susan Murphy 1,2 Institute for Social Research.
An Experimental Paradigm for Developing Adaptive Treatment Strategies S.A. Murphy NIDA Meeting on Treatment and Recovery Processes January, 2004.
Principles of Effective Drug Addiction Treatment Health 10 The Truth About Drugs Ms. Meade.
Designing An Adaptive Treatment Susan A. Murphy Univ. of Michigan Joint with Linda Collins & Karen Bierman Pennsylvania State Univ.
SMART Trials for Developing Adaptive Treatment Strategies S.A. Murphy Workshop on Adaptive Treatment Designs NCDEU, 2006.
CLINICAL PROTOCOL DEVELOPMENT
Presentation transcript:

1 Meeting the Future in Managing Chronic Disorders: Individually Tailored Strategies S.A. Murphy Herbert E. Robbins Collegiate Professorship in Statistics Lecture November 14, 2006

2 Outline –Three apparently dissimilar problems –Myopic decision making –Constructing strategies –Challenges Unknown, unobserved causes Small, expensive data sets –Discussion

3 Three Apparently Dissimilar Problems –Artificial Intelligence: Autonomous Helicopter Flight –Management of Chronic Mental Illnesses –Management of a Welfare Program

4 Artificial Intelligence Autonomous Helicopter Flight –Observations: characteristics of the helicopter (position, orientation, velocity, angular velocity, ….), characteristics of the environment (wind speed, wind angle, turbulence….) –Actions/treatments: cyclic pitch (causes forward/backward and sideways acceleration), tilt angle of main rotor blades (direction), tail rotor pitch control (turning) –Rewards: Closeness of helicopter’s flight path to the desired path; avoidance of crashes(!)

5 Andrew Ng’s Helicopter:

6 The Management of Chronic Mental Illnesses Treating Patients with Opioid Dependence (heroin) –Observations: individual characteristics (withdrawal symptoms, craving, attendance at counseling sessions, results of urine tests….), characteristics of the environment (housing, employment.…) –Actions/treatments: methadone dose, amount of weekly group counseling sessions, daily dosing time of methadone, individual counseling sessions, methadone taper –Rewards: minimize opioid use and maximize health/functionality, minimize cost

7

8 Management of a Welfare Program “Jobs First” Program in Connecticut –Observations: individual characteristics (assets, income, age, health, employment), characteristics of the environment (domestic violence, incapacitated family member, # children, living arrangements…) –Actions/treatments: child care, job search skills training, amount of cash benefit, medical assistance, education –Rewards: maximize employment/independence.

9

10 The Common Thread: Multi-Stage Decision Making Observation, action, observation, action, observation, action,……………………. A strategy tells us how to use the observations to choose the actions. We’d like to develop strategies that maximize the rewards.

11 Myopic Decision Making

12 Myopic Decision Making In myopic decision making, decision makers use strategies that seek to maximize immediate rewards. Problems: –Ignore longer term consequences of present actions. –Ignore the range of feasible future actions/treatments –Ignore the fact that immediate responses to present actions may yield information that pinpoints best future actions (A strategy tells us how to use the observations to choose the actions.)

13 Autonomous Helicopter Flight The helicopter has veered from flight plan. Myopic action: Choose an acceleration and direction that will ASAP bring us back to the flight plan. The result: The myopic action results in the helicopter overshooting the planned flight path and in drastic situations may lead to the helicopter cycling out of control. The mistake: We did not consider the range of actions we can take following the initial action. The ability to slow down is mechanically limited. The message: Use an acceleration that will not return us as quickly to the planned flight path but will take into account the ability of the helicopter to slow down and reduce the overshoot.

14 Treatment of Psychosis Myopic action: Choose a treatment that reduces psychosis for as many people as possible. The result: Some patients are not helped and/or experience abnormal movements of the voluntary muscles (TDs). The class of subsequent medications is greatly reduced. The mistake: We should have taken into account the variety of treatments available to those for whom the first treatment is ineffective. The message: Use an initial medication that may not have as large a success rate but that will be less likely to cause TDs.

15 Treatment of Opioid Dependence Myopic action: Choose an intensive multi-component treatment (methadone + counseling + behavioral contingencies) that immediately reduces opioid use for as many people as possible. The result: Behavioral contingencies are burdensome/expensive to implement and many people may not need the contingencies to improve. The mistake: We should allow the patient to exhibit poor adherence prior to implementing the (negative) behavioral contingencies. The message: Use an initial treatment that may not have as large an immediate success rate but carefully monitor patient adherence to ascertain if behavioral contingencies are required.

16 Constructing Strategies

17 Basic Idea for Constructing a Strategy: Move Backwards Through Time. (Pretend you know the distribution of all outcomes!)

18 Challenges

19 Goal Combine theory and data in a principled fashion to construct a good strategy. In A.I. scientists combine mechanistic theories with data from experiments to construct strategies that maximize the rewards.

20 Artificial Intelligence Scientists who construct strategies in autonomous helicopter flight use mechanistic theory (physical laws: momentum=m*v, W=F*d*cos(θ)…) to model the interrelationships between observations. –Scientists know many (most?) of the causes of the observations and know how the observations relate to one another. Scientists can quickly evaluate the strategies (within a matter of months).

21 Comparatively Less Known Mechanistic Models in Behavioral/Social/Medical Sciences Scientists who want to use data on individuals to construct treatment strategies must confront the fact that non-causal “associations” occur due to the unknown causes of the observations.

22 Conceptual Structure in the Behavioral/Social/Medical Sciences

23 Unknown, Unobserved Causes (Incomplete Mechanistic Models)

24 Unknown, Unobserved Causes (Incomplete Mechanistic Models) Problem: Non-causal associations between treatments (here counseling) and rewards are likely. Solution: Construct strategies using data sets in which randomization is used to assign treatments to students. This breaks the non-causal associations yet permits causal associations.

25 Unknown, Unobserved Causes (Incomplete Mechanistic Models)

26 Unknown, Unobserved Causes (Constructing Sequences of Treatment)

27 Unknown, Unobserved Causes (Incomplete Mechanistic Models)

28 Unknown, Unobserved Causes (Incomplete Mechanistic Models)

29 The problem: Even when treatments are randomized, non-causal associations occur in the data. The solution: Statistical methods for constructing strategies must be conducted in stages as opposed to “all-at-once.” Statistical methods should appropriately “average” over the non-causal associations between treatment and reward. Unknown, Unobserved Causes (Incomplete Mechanistic Models)

30 Summary of Solutions To Causal Problems Subjects in your data should be representative of population of subjects. Experiments should randomize actions. Develop statistical methods that avoid being influenced by non-causal associations yet help you construct the strategy.

31 Expensive Data on a Limited Number of Individuals Scientists who want to use data on individuals to construct treatment strategies must provide measures of confidence and also evaluations of alternative treatment strategies. Methods for constructing strategies are non- smooth.

32 Basic Idea for Constructing a Strategy: Move Backwards Through Time.

33 Expensive, Limited Data on Individuals In order to provide measures of confidence and comparisons of strategies, the statistical methods for constructing strategies must be regularized. A number of statisticians are working hard on this open question. This problem will be one of the foci of the SAMSI program in June, 2007:

34 Some Experiments

35 ExTENd Ongoing study at U. Pennsylvania (D. Oslin) Goal is to learn how best to help alcohol dependent individuals reduce alcohol consumption.

36

37 Adaptive Treatment for ADHD Ongoing study at the State U. of NY at Buffalo (B. Pelham) Goal is to learn how best to help children with ADHD improve functioning at home and school.

38 ADHD Study B. Begin low dose medication 8 weeks Assess- Adequate response? B1. Continue, reassess monthly; randomize if deteriorate B2. Increase dose of medication with monthly changes as needed Random assignment: B3. Add behavioral treatment; medication dose remains stable but intensity of bemod may increase with adaptive modifications based on impairment No A. Begin low-intensity behavior modification 8 weeks Assess- Adequate response? A1. Continue, reassess monthly; randomize if deteriorate A2. Add medication; bemod remains stable but medication dose may vary Random assignment: A3. Increase intensity of bemod with adaptive modifi- cations based on impairment Yes No Random assignment:

39 STAR*D This trial is over and the data is being analyzed (PI: J. Rush). One goal of the trial is construct good treatment sequences for patients suffering from treatment resistant depression.

40

41 Discussion The best management of chronic disorders (poverty, mental illness, other medical conditions) requires multi-stage decision making. Avoid myopic decision making! –Allow for longer term effects of the treatment –When comparing treatment options take into account the effect of future treatments –Appreciate the value of observing patients outcomes such as adherence Experimental designs and statistical methods are available.

42 This seminar can be found at:

43 Role of the Statistician What kinds of data are most useful for developing strategies? How do we use limited and expensive data to construct good strategies? How do we evaluate strategies using the limited data? (A strategy tells us how to use the observations to choose the actions.)

44 Unknown, Unobserved Causes

45 Unknown, Unobserved Causes

46 Unknown, Unobserved Causes Problem: We recruit students via flyers posted in dormitories. Associations between observations and rewards are highly likely to be (due to the unknown causes) non- representative. Solution: Sample a representative group of college students.