The good, the bad, and the ugly of multiple data sources Barbara J. McMorris, PhD ONRS Seminar May 19, 2011 UMN School of Nursing

Types of Triangulation Types of Triangulation MethodologicalAnalysisInterdisciplinaryInvestigatorData

Why is Data Triangulation Important? Better research study & results Reviewers getting savvier; asking for multiple sources of data More “tools” for your research tool box

Study 1: Administrative Claims Data + Self Reports

Rhinitis Study Goals Assess burden of allergic and non allergic rhinitis –What is the cost and utilization impact of treatment for rhinitis? Evaluate providers’ perceptions of rhinitis burden on patients and severity of symptoms

Rhinitis Survey Data Analysis % Agree/Strongly Agree

Rhinitis Claims Data Analysis Count of and Costs of Health Care Services Utilized during 1 Yr Follow-up Mean (SD) Mild Patients (N = 5,670) Moderate Patients (N = 22,216) Severe Patients (N = 7,217) Total (N=35,103) Number of Rhinitis-Related ER Visits 0.03 (0.75)0.05 (0.89)0.04 (0.69)0.05 (0.83) Number of Rhinitis-Related Office Visits 5.61 (10.1)8.12( 11.69)9.39 (12.85)7.98 (11.75) Number of Rhinitis-Related Specialist Office Visits 3.08 (8.50)5.65 (10.43)7.22 (11.79)5.56 (10.52) Total Rhinitis-Related Medical Cost $358 (725)$514 (919)$556 (952)$497 (900) Total Rhinitis-Related Pharmacy Costs $364 (417)$435 (474)$440 (478)$424 (467)

Reflection: “good-bad” Known sampling frame “Clean” processed data Timing of data collection different –Lag in health care claims being loaded –Physical report of severity Low response rate

Results used in marketing /2007_10_19_GSK1134.htm /2007_10_19_GSK1134.htm

Study 2: Self Reports + Provider Care Flow Sheets

RA Infusion Study Goals Describe patient-reported outcomes due to bDMARD infusions: –satisfaction –lost productivity –pain/discomfort Quantify the amount of time providers spend for initiating infusions (3 types) Describe provider satisfaction with infusions

Data Sources Health care claims database Provider survey Patient Survey Case Report Form Infusion Care Flow Sheet

RA Infusion Results Patient and Provider time associated with infusions Means & %’s Infusion 1 (N = 102) Infusion 2 (N = 74) Infusion 3 (N = 29) Total patient time away from work/home (min.) % patients able to return to work45%17%29% % had friend/family member miss work 31%24%43% Total provider labor time (min.)

Reflection: “good” Good example of triangulation Comprehensive picture of “burden” Small sample size

Results used in marketing arthritis-medication/orencia-infusion.aspx arthritis-medication/orencia-infusion.aspx

Study 3: Self Reports + Process Data + Official Records

Broadway HS Study Goals Provide a descriptive profile of pregnant and parenting teen girls Compare outcomes for Broadway and comparison girls:  progress toward graduation  delaying a repeat birth  keeping children fully immunized

Data Sources Girls’ self reports on pre/post surveys Case manager reports of time Official Records –Program –School district –County

Evaluation Results %

County Data MFIP and Other Assistance Broadway Students (N = 158) Comparison Group (N = 97) Ave counted months of MFIP Ave used months of MFIP or more children27% Used emergency assistance47%41% Case closed due to sanction limit9%13%

Reflection: kinda “ugly” Using survey created/revised by others Linking datasets – what is the link? Consent to access official records? What were the data collected for? –Who entered the data? Budget enough time & money for both data management & analysis?

Final Cautions about Data Triangulation Need well-defined goals (no “fishing”) Costly both in time and $$$ May not solve bias/error issues Replication is difficult Know your “official” records sources and their limitations Do you have “permission” to link data sets?

“triangulation state of mind”* “If you self consciously set out to collect and double-check findings using multiple sources and modes of evidence, the verification process will largely be built into the data-gathering process, and little more need be done than to report on one’s procedures.” *Miles & Huberman (1984: p.235)

Thank you! Useful citations: Begley, CM. (1996). Using triangulation in nursing research. Journal of Advanced Nursing 24: Thurmond, VA. (2001). The Point of Triangulation. Journal of Nursing Scholarship 22(3):