1. How useful is a reminder system in collection of follow-up quality of life data in clinical trials?
Dr Shona Fielding

2. Outline Background Example datasets
What is quality of life (QoL)?
Missing data
Missing data mechanism
Ways of dealing with missing data
Example datasets
The reminder-system
Comparison of methods using reminder data
Conclusions

3. What is quality of life? Measure of health status
Dimensions includes physical functioning, mental functioning, social well-being, cognitive functioning, pain
Generic measures
EuroQoL (EQ5D): overall health status
(mobility, self-care, usual activities, pain/discomfort, anxiety)
Disease specific measures
Oxford Knee Score (OKS)
QLQ-C30 cancer specific questionnaire

4. Missing data
Data you expect to collect: e.g. non-response to a postal questionnaire
Missing forms or Missing Items
Why is it a problem?
Loss of power due to reduced sample sizes
Introduce bias to results
Serious problem in analysis of QoL outcomes, as missing data likely to be informative

5. Missing data mechanism
Three missing data mechanisms
Missing completely at random (MCAR)
Missing at random (MAR)
Missing not at random (MNAR)
Dependent on…
Independent of…
MCAR
Covariates
Observed QoL
Missing QoL
MAR
MNAR
Table 1: Simple overview of missing data mechanisms

6. Dealing with missing data
Complete case analysis
Imputation
Simple imputation: single value e.g. last value carried forwards (LVCF), mean imputation
Multiple imputation: several values, incorporates uncertainty
Model-based procedures
e.g. pattern mixture model (not discussed here)

7. Example Trials RECORD trial (N=5292): REFLUX (N=357)
Vitamin supplementation in the elderly to prevent re-fracture
Treatment comparisons: Calcium versus No calcium
QoL: EQ5D
Analysis of covariance (ANCOVA) at 24 months adjusting for baseline QoL plus other patient variables
REFLUX (N=357)
Comparison of surgery with medical management for gastro-oesophageal reflux disease
ANCOVA at 12 months adjusting for baseline QoL, sex, age, BMI
QoL: Reflux Quality of Life Score (RQLS)

8. The reminder system
Each trial used reminder system for follow-up questionnaires, leading to three types of responders
Immediate responders (respond to initial mailing)
Reminder responders (respond following reminder)
Non-responders (not sent the questionnaire or did not return questionnaire)

9. Table 2: Response rate foe example trials
Response Rates
RECORD N(%)
REFLUX N(%)
Immediate responders
2695 (51%)
136 (38%)
Reminder responders
741 (14%)
183 (51%)
Non-responders
1856 (35%)
38 (11%)
Table 2: Response rate foe example trials
Overall response rate (including reminders)
RECORD – 65% ; REFLUX – 89%

10. What did we do?
Use the ‘extra’ data (from reminder responses) to test procedures for dealing with missing data
Investigated the missing data mechanism
Investigated suitable imputation procedures

11. Missing data mechanism
Several methods used:
Two hypothesis tests
Two logistic regression procedures
RECORD
Non-response: MAR
Reminder response: MAR
REFLUX
Non-response: MCAR
Reminder response: MCAR
Ref: Shona Fielding, Peter M Fayers, Craig R Ramsay. Investigating the missingness mechanism in
quality of life data: A comparison of approaches. Health and Quality of Life Outcomes 2009, 7: 57.

12. Investigating suitable imputation methods
Subset of responders was identified
Reminder-responses removed → imputation carried out → ANCOVA on imputed dataset
Result compared to the original ANCOVA result
Identify the most suitable method of imputation

13. Imputation of reminder responses
RECORD
LVCF was ‘best’ simple imputation method
Predictive mean match model was ‘best’ multiple imputation model
Imputation method did impact on the trial conclusion
REFLUX
Imputation method affected the magnitude of treatment difference estimate but not the trial conclusion

14. Comparing analysis strategies
How does the choice of analysis strategy affect the result?
Different strategies considered
ANCOVA on immediate responses
ANCOVA on all responses (including reminder)
Repeated measures on immediate responses
Repeated measures on all responses
LVCF following immediate responses
LVCF following all responses
MI (predictive mean match) following immediate responses
MI (predictive mean match) following all responses

15. RECORD – different analysis strategies
Method N
Difference (95% CI)
P-value 1
Immediate responses (ANCOVA)
1919 (36%)
(-0.005, 0.03)
0.16 2
All responses (ANCOVA)
2879 (54%)
(0.00, 0.30)
0.05 3
Immediate responses
(repeated measures)
2907 (55%)
(0.0004, 0.03)
0.04 4
All responses
3906 (74%)
(0.003, 0.03)
0.02 5
Immediate responses + LVCF
(0.003, 0.28) 6
All responses + LVCF
(0.004, 0.28)
0.01 7
Immediate responses + MI
5292 (100%)
(-0.003, 0.02)
0.15 8
All responses + MI
(-0.003, 0.02)
0.14
Table 3: Treatment Difference (95% CI) in EQ5D scores for different analysis strategies

16. RECORD

17. REFLUX– different analysis strategies
Method N
Difference (95% CI)* 1
Immediate responses (ANCOVA)
121 (34%)
12.9 (6.6,19.2) 2
All responses (ANCOVA)
276 (77%)
14.1 (9.5,18.6) 3
Immediate responses (repeated measures)
327 (92%)
12.6 (6.8,18.4) 4
All responses (repeated measures)
14.1 (9.7,18.5) 5
Immediate responses + LVCF
6.5 (2.9,10.6) 6
All responses + LVCF
11.9 (7.7,16.1) 7
Immediate responses + MI
342 (96%)
11.6 (5.7,17.5) 8
All responses + MI
353 (99%)
13.0 (8.9,17.0)
* All p<0.001
Table 4: Treatment Difference (95% CI) in RQLS scores for different analysis strategies

18. REFLUX

19. What should you do?
No single way of dealing with missing data that is applicable in all situations
Plan study to minimise missing data
Use REMINDERS for follow up questionnaires

20. What should you do? If still have missing data then
Identify the missing data mechanism
MCAR – complete case or simple imputation may be used
MAR – repeated measures or multiple imputation
MNAR – model-based strategy
Make use of the reminder data to help inform which particular method of imputation (if any) is appropriate

21. Conclusion
A reminder system extremely useful way of recovering data originally missing
It is a cost effective use of resources to maintain the sample size
Using reminders to minimize the amount of missing data also reduces the threat of bias
Data collected by reminders enables a more informed selection of imputation methods, which again reduces the risk of bias

22. Acknowledgements Health Services Research Unit (HSRU) for
providing the data
HSRU is funded by the Chief Scientist Office (CSO)
of the Scottish Government Health Directorate
CSO for funding my Research Training Fellowship
(CZF/1/31)
Project supervisors Professor Peter Fayers
and Dr Craig Ramsay
Others: Jonathan Cook, Graeme Maclennan, Cynthia Fraser, Luke Vale, Samanthan Wileman, Janice Cruden, Gladys McPherson, Alison Macdonald, Seonaidh Cotton
(All University of Aberdeen)

23. ANY QUESTIONS?