1. Session: The Integration of Care for Persons with Disabilities
Using Comparative Effectiveness Research and Creative Orthogonal Designs to Improve Outcomes and Increase Efficiency
Randall Brown
Session: The Integration of Care for Persons with Disabilities
2010SNP Leadership Forum
October 28, 2010

2. Background
HHS-OD interested in funding comparative effectiveness research to improve outcomes for dual eligible adults with disabilities
Contracted with Mathematica team, experienced evaluator of care coordination programs
SNP Alliance/NHPG interested in improving outcomes
Dr. Kieron Dey’s pathbreaking work on orthogonal design

3. The Problem
Many components of effective CC/CM for members with disabilities:
Assessment
Care planning
Monitoring
Triaging
Patient education
Facilitating transitions
Improving communication among providers, and between patients and providers
Increasing patient adherence
Service arrangement
Medication management

4. The Problem (continued)
Many ways to implement these components
Frequency/mode of CC/CM contacts with patient/provider
Composition of care team
Frequency and mode of team meetings
Thresholds for identifying patients to get more intensive intervention
Standardized protocol only vs. CC/CM judgment
Assessment tool used
Screening tool for problems (e.g., for depression)
Motivational interviewing
Strengthen component (Rx review, cg support, etc.)
Little evidence on how to make these choices
More of an intervention isn’t necessarily better

5. Testing What Works
Want rigorous, valid, replicable test of what works best
Adequate sample size essential to ensure low likelihood of accepting false theories AND low likelihood of rejecting true ones
Usual approach: test one variant per RCT study
Takes forever, expensive
Doesn’t account for interactive effects
Efficient orthogonal designs overcome these problems
Allows testing many intervention variants simultaneously

6. What is Efficient Orthogonal Design?
Developed by famous statistician (RA Fisher) long ago
Used in many industries, but rarely in health care
Ingenious, powerful method of testing relative effectiveness of 2 alternatives for many intervention components simultaneously, without requiring every possible combination
Identify interventions to test (n, say)
Identify units of intervention (e.g., CC/CM)

7. What is Efficient Orthogonal Design?
Use algorithm to determine number of intervention units required and precise combination of n interventions to assign to each operation unit
For each intervention, half of CC/CM implement option A, half option B
But different half for different interventions
Each CC/CM randomly assigned to particular intervention mix
Measure outcomes of interest for each patient
Compute difference of mean outcome for each intervention (YA – YB)

8. Illustrative Example
4 interventions to test (e.g., caseload, assessment tool, pharmacist on care team, minimum # contacts/mo.)—binary choices
Testing all combinations of the 4 interventions (i.e., full factorial design) would require 16 CC/CMs
More efficient orthogonal designs require fewer CC/CMs
Fractional factorial would take only 8 CC/CMs
Gains much more dramatic when testing many interventions: (e.g., Plackett-Burman design requires only 20 CC/CMs to test 19 interventions)

9. Illustrative Example (continued)
Interventions to be tested (A=basic; B=enhanced):
Caseload: 100 (option A) vs. 70 (option B)
Assessment tool: simple form (A) vs. longer form (B)
Pharmacist: available for consult (A) vs. on CM team (B)
Minimum # contacts: 1/mo. (A) vs. 2/mo (B)
Assign 8 nurses randomly to these 8 choices:
AAAA, BAAB, ABAB, BBAA, AABA, BABB, ABBB, BBBA
So, the nurse assigned to combination ABAB would have high caseload, use a longer assessment tool, have a pharmacist available for consult, and contact patients at least twice per month

10. Current Study in Planning Stage
Study limited to dual eligibles with disabilities
Focusing on SNPs with comparable structures
Design for SNPs with some CC/CM in-person contact
Adapt if possible for telephonic only
Goal is 5-9 SNPs, patients, 20+ interventions
Survey to capture patient-centered outcomes
Also measure effects on service use, quality of care, costs
Study to be completed before May 2012

11. What Will SNPs Get from the Study?
If intervention options well-chosen and sample size adequate, CAN’T FAIL TO LEARN SOMETHING USEFUL
Statistically significant findings reveal better options
Insignificant findings can identify less expensive choices that are equally effective
Answers to questions SNPs have identified as important re whether/how to enhance interventions
Quick, rigorous results on patient well-being, satisfaction, quality of care, service use, costs
Enhanced reputation for pursuing high quality care

12. Study Challenges Agreeing on intervention options to test
Some options aren’t acceptable to some plans
But creative collaboration between plans and researchers can identify key testable options
Overcoming fears of complexity
Training CC/CMs on interventions
Need enough cases per CC/CM to justify
IRB approval required? (operations waiver possible)
OMB approval needed for survey

13. Future Opportunities
More efficient orthogonal designs enable CER on many issues of importance to SNPs
ACTION grants are a potential funding source
MPR and SNP Alliance are partnering (AHRQ contract)
Contact me with ideas you would like to pursue