Combined predictor Selection for Multiple Clinical Outcomes Using PHREG Grisell Diaz-Ramirez

Background PHREG procedure has built-in options to perform predictor selection using stepwise methods or best subsets Built-in options for predictor selection do not allow the simultaneous selection of predictors for multiple outcomes The selection of a common set of predictors for multiple outcomes is important in clinical settings: Practitioners want to evaluate the effects of risk-factors on multiple health-related outcomes in the same study

Background (cont.) There are many variable-selection algorithms for individual outcomes, but there are limited studies on the selection of a common set of predictors for multiple outcomes Information Criteria like the Bayesian Information Criterion (BIC) and stepwise regression methods (e.g. backward elimination) are commonly used when selecting the best set of predictors for individual outcomes

Objectives We describe a SAS Macro for selecting a common set of variables for predicting multiple clinical outcomes using PHREG The selection method uses a variant of backward elimination based on the average normalized-BIC across multiple outcomes We test our method using the Health and Retirement Study (HRS) data Compare final predictor set from BIC Combined Backward Elimination with final predictor sets from BIC Backward Elimination by Outcome Perform simulation study

Methods Performed BIC backward elimination using a SAS macro we developed Data: HRS Outcomes (4 time-to-event clinical outcomes from HRS): Time to first Activity of Daily Living (ADL) dependence Time to first Instrumental Activity of Daily Living (IADL) difficulty Time to first walking-across-the-room dependence Time to death Predictors: 39 health-related and demographic predictors measured at baseline from HRS Measured predictive accuracy using the Harrell’s C-statistic for all fitted models

Methods (cont.) Fitted survival regression models: Fine and Gray Competing-risk regression: time to 1st ADL, IADL, walking-across the room. When there are multiple types of events, a competing risk event (e.g. death) can modify the chance that the event of interest occurs (e.g. first ADL), so we need to appropriately account for it Cox regression: time to death. There is only one type of event, namely all-cause mortality

Normalized BIC (k)= 𝐵𝐼𝐶(𝑘) (𝐵𝐼𝐶 𝑓𝑢𝑙𝑙 𝑚𝑜𝑑𝑒𝑙 −𝐵𝐼𝐶 𝑏𝑒𝑠𝑡 𝑖𝑛𝑑𝑖𝑣𝑖𝑑𝑢𝑎𝑙 𝑚𝑜𝑑𝑒𝑙) Methods (cont.) Used normalized BIC: Changes in BIC from complex models to simpler models count approximately the same across multiple outcomes: Normalized BIC (k)= 𝐵𝐼𝐶(𝑘) (𝐵𝐼𝐶 𝑓𝑢𝑙𝑙 𝑚𝑜𝑑𝑒𝑙 −𝐵𝐼𝐶 𝑏𝑒𝑠𝑡 𝑖𝑛𝑑𝑖𝑣𝑖𝑑𝑢𝑎𝑙 𝑚𝑜𝑑𝑒𝑙) BIC=−2𝑙𝑜𝑔𝐿+𝑘 log 𝑛 L: the maximized value of the likelihood function of the fitted model k: number of parameters estimated by the fitted model n: sample size

Objectives We describe a SAS Macro for selecting a common set of variables for predicting multiple clinical outcomes using PHREG The selection method uses a variant of backward elimination based on the average normalized BIC across multiple outcomes We test our method using the Health and Retirement Study (HRS) data Compare final predictor set from BIC Combined Backward Elimination with final predictor sets from BIC Backward Elimination by Outcome Perform simulation study

SAS Macro For BIC Combined Backward Elimination Select final set of predictors with minimum average normalized BIC from p-1 to 2 predictors Same process is repeated until there are two variables left in the model Last model only with variables that are foced in (e.g. age and sex) Start with model with p-1 predictors (e.g. 38) Select subset of p-2 predictors with minimum average normalized BIC (e.g. p=37) Start with Full model with all predictors (p) (e.g p=39) Select subset of p-1 predictors with minimum average normalized BIC (e.g. p=38)

Objectives We describe a SAS Macro for selecting a common set of variables for predicting multiple clinical outcomes using PHREG The selection method uses a variant of backward elimination based on the average normalized BIC across multiple outcomes We test our method using the Health and Retirement Study (HRS) data Compare final predictor set from BIC Combined Backward Elimination with final predictor sets from BIC Backward Elimination by Outcome Perform simulation study

Case Study: HRS data Nationally representative cohort of 5,531 community- dwelling seniors enrolled in HRS: Longitudinal survey of a representative sample of all persons in the United States over age 50. Interviews are every two years on subjects like health care, housing, assets, pensions, employment, and disability Included respondents 70-years old or older at the time of their interview in 2000 The follow-up time was up to 2014

Case Study: HRS data (cont.) No. Variables Average Normalized-BIC Average C-statistic 39 134.271 0.669 38 134.181 37 134.094 0.668 … 14 133.492 0.655 4 133.930 0.632 3 134.110 0.628 2 134.410 0.611 In the red box is the final model with the minimum average normalized-BIC

Objectives We describe a SAS Macro for selecting a common set of variables for predicting multiple clinical outcomes using PHREG The selection method uses a variant of backward elimination based on the average normalized BIC across multiple outcomes We test our method using the Health and Retirement Study (HRS) data Compare final predictor set from BIC Combined Backward Elimination with final predictor sets from BIC Backward Elimination by Outcome Perform simulation study

BIC Backward Elimination by Individual Outcome Ran the BIC backward elimination macro for each individual outcome and selected a final model for each outcome with the minimum BIC Created a set of predictors with the union of all predictors in the final model of each outcome Compared it with the set of predictors obtained using combined outcomes

BIC Backward Elimination by Individual Outcome (cont.) Harrell's C-statistic time to first ADL dependence (9 variables) time to first IADL difficulty (8 variables) time to first walking-across-the-room dependence (7 variables) time to death (16 variables) Average Individual models 0.637 0.635 0.636 0.711 Combined model by outcome (14 variables) 0.639 0.642 0.706 0.655 (average 4 models) Union model by outcome (22 variables) 0.647 0.638 0.648 0.661 (average 4 models) Combined model is more parsimonious with 14 variables vs. 22 variables in union model C-statistic in the combined model by outcome is very similar to the predictive accuracy of individual and union models Final combined model has a good balance between parsimony and predictive accuracy

Objectives We describe a SAS Macro for selecting a common set of variables for predicting multiple clinical outcomes using PHREG The selection method uses a variant of backward elimination based on the average normalized BIC across multiple outcomes We test our method using the Health and Retirement Study (HRS) data Compare final predictor set from BIC Combined Backward Elimination with final predictor sets from BIC Backward Elimination by Outcome Perform simulation study

Simulation Study Created two sets of 100 simulated data sets with the same sample size (N=5,531) and number of initial predictors (i.e. 39) Generated one set of 100 simulated data sets with high correlation among the outcomes and a second set of 100 simulated data sets with low correlation among the outcomes Ran the BIC combined backward elimination macro in each of the simulated data sets Computed the percentage of correct inclusion and the percentage of correct exclusion of predictors in the final simulated models

Simulation Study (cont.) Variable Percentage Correct Inclusion Uncorrelated outcomes Percentage Correct Inclusion Correlated outcomes COGDLRC3G 100 DIABETES DRIVE 95 89 EDU 78 71 EXERCISE 93 91 HEARTFAILURE 94 96 INCONTINENCE 81 LUNG 73 74 OTHERCLIM3G 87 82 OTHERPUSH 97 92 SMOKING VOLUNTEER 98 Mean 91.33 89.25 Most of the predictors have a percentage of correct inclusion of 80% or higher. That is, they show up in 80-100% of the final combined models using simulated data sets Percentage of correct exclusion is almost 100% (result not shown in table)

Simulation Study (cont.) Harrell's C-Statistic No. Variables Final Combined Model Original data set 0.655 14 Simulated data sets with uncorrelated outcomes (mean [95% CI]) 0.647 [0.646-0.647] 12.960 [12.793-13.127] Simulated data sets with correlated outcomes (mean [95% CI]) 0.646 [ 0.645-0.648] 12.720 [12.548-12.892] Computed mean and 95% Confidence Interval (CI) of the C-statistic and the number of predictors in the final combined models from simulated data sets C-statistic of the final combined models from the simulated data sets is very similar to the C-statistic of the final combined model in the original data Means of the number of predictors in the final combined models from simulated data sets are slightly smaller

Conclusions It is possible to select a common set of predictors for multiple clinical outcomes using PROC PHREG and a variant of backward elimination based on the average normalized Bayesian Information Criterion (BIC) across outcomes Our method provides a straightforward approach without compromising parsimony or predictive accuracy This method could also be applied to continuous or categorical outcome variables, and other Information Criteria like the AIC can be used

Contact Information Name: Grisell Diaz-Ramirez Company: University of California, San Francisco Email: grisell.diaz-ramirez@ucsf.edu