Finite Patient Horizon NYSPI1 Design of Local Investigations in Community Practice Settings: An Effectiveness / Cost-Effectiveness Framework with Finite Patient Horizon Naihua Duan, Ph.D. Columbia University Joint work with Ken Cheung (Columbia University) and Jeff Cully (Houston VA HSRD Center)

Finite Patient Horizon NYSPI2 Why Local Investigations? Think globally, act locally Quality improvement programs in community practice settings often need to adapt and/or improvise in order to accommodate local conditions Impact of adaption is usually unknown Decision-making often based on expert judgment Empirical evaluation might be warranted, but not widely used –“Small n problem” (Small N, not n)

Finite Patient Horizon NYSPI3 External Program Evaluation vs. Internal Quality Improvement Evaluation Program evaluation usually takes an external perspective: does the implementation program work? –Does the engine work overall? Internal evaluation for nuts and bolts might be warranted for quality improvement –How do we do intake calls, case referrals, follow-up reminders, care management (maybe IT enhanced), etc., most effectively for a specific clinical setting? One size might not fit all (HTE; Kravitz-Duan 2004)

Finite Patient Horizon NYSPI4 Statistical Literature Statistical literature focused mainly on global investigations, not on local investigations –Zelen (1969) an unusual exception “New” statistical perspective warranted for local investigations in implementation studies Global knowledge for external consumption and local knowledge for internal consumption call for distinct statistical frameworks

Finite Patient Horizon NYSPI5 Patient Horizon (Population Size) Essentially infinite for usual clinical studies aimed to produce generalizable knowledge for external consumption –Large Phase III trial, sample size << N –Strong accuracy (small , high power) warranted to protect welfare of future patients Can be very limited (hundreds, even fewer) for local investigations aimed to produce local knowledge for internal consumption

Finite Patient Horizon NYSPI6 Hypothetical Example N = 500 patients Randomize 2n patients to novel procedure vs. standard procedure (50/50) Apply local knowledge gained to remaining patients (N – 2 n) d = 0.25,  = 5%, 80% power ==> n = 250 No patients left to consume knowledge gained!

Finite Patient Horizon NYSPI7 Cost Effectiveness Framework Criterion: incremental net benefit per capita –INB = b  – c –  = effect size, say, symptom reduction resulting from new procedure (unknown) –b = value for each unit of treatment effect (known) –c = incremental cost for novel procedure vs. standard procedure (known) Effectiveness framework if c = 0

Finite Patient Horizon NYSPI8 Cost Effectiveness Framework (II) Bayesian framework to incorporate existing knowledge among local experts –  ~ N(  0,  2 ) –  measures local experts’ uncertainty Cost-effectiveness equipoise: –b  0 = c –Effectiveness equipoise:  0 = 0 (c = 0)

Finite Patient Horizon NYSPI9 Non-Empirical Adoption Strategies Based adoption decision on expert opinion, without empirical local investigation Never adopt strategy (bench mark) Always adopt strategy –Expected gain compared to never adopt: –G1 = N × (b  0 – c)

Finite Patient Horizon NYSPI10 Evidence-Based Adoption Strategy Randomize 2n patients to trial (evaluation phase) One-sided test H 0 : INB  0 vs. H A : INB > 0 with significance threshold  Apply finding to remaining N – 2n patients (consumption phase) Each combination of design parameters (n,  ) is a possible design for the local investigation Find the design that maximizes expected net gain across N patients Compare with non-empirical strategies

Finite Patient Horizon NYSPI11 Evidence-Based Adoption Strategy Expected gain: G2 = {n + (N – 2n) A} (b  0 – c) + (N – 2n) B A  (0,1); B > 0 Compared to non-empirical strategies: G0 = 0 G1 = N × (b  0 – c)

Finite Patient Horizon NYSPI12 Optimal Design for Local Investigation

Finite Patient Horizon NYSPI13

Finite Patient Horizon NYSPI14 Noteworthy Results Under equipoise: –Any local investigation is better than non-empirical strategies –Optimal  = 50% –Optimal n = N / {3 + sqrt(9 + 4 R)}  N / 6 R = N  2 / (2  2 ) Under mild optimism: optimal  > 50% Under strong optimism: forthright adoption without local investigation

Finite Patient Horizon NYSPI15

Finite Patient Horizon NYSPI16 Discussions More emphasis on local investigations might enhance quality improvement programs, leading to improved patient safety, quality of care, and health outcomes More emphasis on local investigations might help transform current top-down organization for health care knowledge production, and facilitate the development of learning communities, and motivate more comprehensive data acquisition

Finite Patient Horizon NYSPI17 Discussions (II) Appropriate statistical methodologies (not the 5%-80% ritual) can facilitate wider use of local investigations in quality improvement studies Human subjects and publication issues