How useful is a reminder system in collection of follow-up quality of life data in clinical trials? Dr Shona Fielding

Outline Background Example datasets What is quality of life (QoL)? Missing data Missing data mechanism Ways of dealing with missing data Example datasets The reminder-system Comparison of methods using reminder data Conclusions

What is quality of life? Measure of health status Dimensions includes physical functioning, mental functioning, social well-being, cognitive functioning, pain Generic measures EuroQoL (EQ5D): overall health status (mobility, self-care, usual activities, pain/discomfort, anxiety) Disease specific measures Oxford Knee Score (OKS) QLQ-C30 cancer specific questionnaire

Missing data Data you expect to collect: e.g. non-response to a postal questionnaire Missing forms or Missing Items Why is it a problem? Loss of power due to reduced sample sizes Introduce bias to results Serious problem in analysis of QoL outcomes, as missing data likely to be informative

Missing data mechanism Three missing data mechanisms Missing completely at random (MCAR) Missing at random (MAR) Missing not at random (MNAR) Dependent on… Independent of… MCAR Covariates Observed QoL Missing QoL MAR MNAR Table 1: Simple overview of missing data mechanisms

Dealing with missing data Complete case analysis Imputation Simple imputation: single value e.g. last value carried forwards (LVCF), mean imputation Multiple imputation: several values, incorporates uncertainty Model-based procedures e.g. pattern mixture model (not discussed here)

Example Trials RECORD trial (N=5292): REFLUX (N=357) Vitamin supplementation in the elderly to prevent re-fracture Treatment comparisons: Calcium versus No calcium QoL: EQ5D Analysis of covariance (ANCOVA) at 24 months adjusting for baseline QoL plus other patient variables REFLUX (N=357) Comparison of surgery with medical management for gastro-oesophageal reflux disease ANCOVA at 12 months adjusting for baseline QoL, sex, age, BMI QoL: Reflux Quality of Life Score (RQLS)

The reminder system Each trial used reminder system for follow-up questionnaires, leading to three types of responders Immediate responders (respond to initial mailing) Reminder responders (respond following reminder) Non-responders (not sent the questionnaire or did not return questionnaire)

Table 2: Response rate foe example trials Response Rates RECORD N(%) REFLUX N(%) Immediate responders 2695 (51%) 136 (38%) Reminder responders 741 (14%) 183 (51%) Non-responders 1856 (35%) 38 (11%) Table 2: Response rate foe example trials Overall response rate (including reminders) RECORD – 65% ; REFLUX – 89%

What did we do? Use the ‘extra’ data (from reminder responses) to test procedures for dealing with missing data Investigated the missing data mechanism Investigated suitable imputation procedures

Missing data mechanism Several methods used: Two hypothesis tests Two logistic regression procedures RECORD Non-response: MAR Reminder response: MAR REFLUX Non-response: MCAR Reminder response: MCAR Ref: Shona Fielding, Peter M Fayers, Craig R Ramsay. Investigating the missingness mechanism in quality of life data: A comparison of approaches. Health and Quality of Life Outcomes 2009, 7: 57.

Investigating suitable imputation methods Subset of responders was identified Reminder-responses removed → imputation carried out → ANCOVA on imputed dataset Result compared to the original ANCOVA result Identify the most suitable method of imputation

Imputation of reminder responses RECORD LVCF was ‘best’ simple imputation method Predictive mean match model was ‘best’ multiple imputation model Imputation method did impact on the trial conclusion REFLUX Imputation method affected the magnitude of treatment difference estimate but not the trial conclusion

Comparing analysis strategies How does the choice of analysis strategy affect the result? Different strategies considered ANCOVA on immediate responses ANCOVA on all responses (including reminder) Repeated measures on immediate responses Repeated measures on all responses LVCF following immediate responses LVCF following all responses MI (predictive mean match) following immediate responses MI (predictive mean match) following all responses

RECORD – different analysis strategies Method N Difference (95% CI) P-value 1 Immediate responses (ANCOVA) 1919 (36%) 0.012 (-0.005, 0.03) 0.16 2 All responses (ANCOVA) 2879 (54%) 0.015 (0.00, 0.30) 0.05 3 Immediate responses (repeated measures) 2907 (55%) 0.016 (0.0004, 0.03) 0.04 4 All responses 3906 (74%) 0.017 (0.003, 0.03) 0.02 5 Immediate responses + LVCF 0.015 (0.003, 0.28) 6 All responses + LVCF 0.016 (0.004, 0.28) 0.01 7 Immediate responses + MI 5292 (100%) 0.009 (-0.003, 0.02) 0.15 8 All responses + MI 0.010 (-0.003, 0.02) 0.14 Table 3: Treatment Difference (95% CI) in EQ5D scores for different analysis strategies

RECORD

REFLUX– different analysis strategies Method N Difference (95% CI)* 1 Immediate responses (ANCOVA) 121 (34%) 12.9 (6.6,19.2) 2 All responses (ANCOVA) 276 (77%) 14.1 (9.5,18.6) 3 Immediate responses (repeated measures) 327 (92%) 12.6 (6.8,18.4) 4 All responses (repeated measures) 14.1 (9.7,18.5) 5 Immediate responses + LVCF 6.5 (2.9,10.6) 6 All responses + LVCF 11.9 (7.7,16.1) 7 Immediate responses + MI 342 (96%) 11.6 (5.7,17.5) 8 All responses + MI 353 (99%) 13.0 (8.9,17.0) * All p<0.001 Table 4: Treatment Difference (95% CI) in RQLS scores for different analysis strategies

REFLUX

What should you do? No single way of dealing with missing data that is applicable in all situations Plan study to minimise missing data Use REMINDERS for follow up questionnaires

What should you do? If still have missing data then Identify the missing data mechanism MCAR – complete case or simple imputation may be used MAR – repeated measures or multiple imputation MNAR – model-based strategy Make use of the reminder data to help inform which particular method of imputation (if any) is appropriate

Conclusion A reminder system extremely useful way of recovering data originally missing It is a cost effective use of resources to maintain the sample size Using reminders to minimize the amount of missing data also reduces the threat of bias Data collected by reminders enables a more informed selection of imputation methods, which again reduces the risk of bias

Acknowledgements Health Services Research Unit (HSRU) for providing the data HSRU is funded by the Chief Scientist Office (CSO) of the Scottish Government Health Directorate CSO for funding my Research Training Fellowship (CZF/1/31) Project supervisors Professor Peter Fayers and Dr Craig Ramsay Others: Jonathan Cook, Graeme Maclennan, Cynthia Fraser, Luke Vale, Samanthan Wileman, Janice Cruden, Gladys McPherson, Alison Macdonald, Seonaidh Cotton (All University of Aberdeen)

ANY QUESTIONS?