1. Combined predictor Selection for Multiple Clinical Outcomes Using PHREG
Grisell Diaz-Ramirez

2. Background
PHREG procedure has built-in options to perform predictor selection using stepwise methods or best subsets
Built-in options for predictor selection do not allow the simultaneous selection of predictors for multiple outcomes
The selection of a common set of predictors for multiple outcomes is important in clinical settings:
Practitioners want to evaluate the effects of risk-factors on multiple health-related outcomes in the same study

3. Background (cont.)
There are many variable-selection algorithms for individual outcomes, but there are limited studies on the selection of a common set of predictors for multiple outcomes
Information Criteria like the Bayesian Information Criterion (BIC) and stepwise regression methods (e.g. backward elimination) are commonly used when selecting the best set of predictors for individual outcomes

4. Objectives
We describe a SAS Macro for selecting a common set of variables for predicting multiple clinical outcomes using PHREG
The selection method uses a variant of backward elimination based on the average normalized-BIC across multiple outcomes
We test our method using the Health and Retirement Study (HRS) data
Compare final predictor set from BIC Combined Backward Elimination with final predictor sets from BIC Backward Elimination by Outcome
Perform simulation study

5. Methods
Performed BIC backward elimination using a SAS macro we developed
Data: HRS
Outcomes (4 time-to-event clinical outcomes from HRS):
Time to first Activity of Daily Living (ADL) dependence
Time to first Instrumental Activity of Daily Living (IADL) difficulty
Time to first walking-across-the-room dependence
Time to death
Predictors: 39 health-related and demographic predictors measured at baseline from HRS
Measured predictive accuracy using the Harrell’s C-statistic for all fitted models

6. Methods (cont.) Fitted survival regression models:
Fine and Gray Competing-risk regression: time to 1st ADL, IADL, walking-across the room. When there are multiple types of events, a competing risk event (e.g. death) can modify the chance that the event of interest occurs (e.g. first ADL), so we need to appropriately account for it
Cox regression: time to death. There is only one type of event, namely all-cause mortality

7. Normalized BIC (k)= 𝐵𝐼𝐶(𝑘) (𝐵𝐼𝐶 𝑓𝑢𝑙𝑙 𝑚𝑜𝑑𝑒𝑙 −𝐵𝐼𝐶 𝑏𝑒𝑠𝑡 𝑖𝑛𝑑𝑖𝑣𝑖𝑑𝑢𝑎𝑙 𝑚𝑜𝑑𝑒𝑙)
Methods (cont.)
Used normalized BIC: Changes in BIC from complex models to simpler models count approximately the same across multiple outcomes:
Normalized BIC (k)= 𝐵𝐼𝐶(𝑘) (𝐵𝐼𝐶 𝑓𝑢𝑙𝑙 𝑚𝑜𝑑𝑒𝑙 −𝐵𝐼𝐶 𝑏𝑒𝑠𝑡 𝑖𝑛𝑑𝑖𝑣𝑖𝑑𝑢𝑎𝑙 𝑚𝑜𝑑𝑒𝑙)
BIC=−2𝑙𝑜𝑔𝐿+𝑘 log 𝑛
L: the maximized value of the likelihood function of the fitted model
k: number of parameters estimated by the fitted model
n: sample size

8. Objectives
We describe a SAS Macro for selecting a common set of variables for predicting multiple clinical outcomes using PHREG
The selection method uses a variant of backward elimination based on the average normalized BIC across multiple outcomes
We test our method using the Health and Retirement Study (HRS) data
Compare final predictor set from BIC Combined Backward Elimination with final predictor sets from BIC Backward Elimination by Outcome
Perform simulation study

9. SAS Macro For BIC Combined Backward Elimination
Select final set of predictors with minimum average normalized BIC from p-1 to 2 predictors
Same process is repeated until there are two variables left in the model
Last model only with variables that are foced in (e.g. age and sex)
Start with model with p-1 predictors (e.g. 38)
Select subset of p-2 predictors with minimum average normalized BIC (e.g. p=37)
Start with Full model with all predictors (p) (e.g p=39)
Select subset of p-1 predictors with minimum average normalized BIC (e.g. p=38)

10. Objectives
We describe a SAS Macro for selecting a common set of variables for predicting multiple clinical outcomes using PHREG
The selection method uses a variant of backward elimination based on the average normalized BIC across multiple outcomes
We test our method using the Health and Retirement Study (HRS) data
Compare final predictor set from BIC Combined Backward Elimination with final predictor sets from BIC Backward Elimination by Outcome
Perform simulation study

11. Case Study: HRS data
Nationally representative cohort of 5,531 community- dwelling seniors enrolled in HRS:
Longitudinal survey of a representative sample of all persons in the United States over age 50. Interviews are every two years on subjects like health care, housing, assets, pensions, employment, and disability
Included respondents 70-years old or older at the time of their interview in 2000
The follow-up time was up to 2014

12. Case Study: HRS data (cont.)
No. Variables
Average Normalized-BIC
Average C-statistic 39
0.669 38 37
0.668 … 14
0.655 4
0.632 3
0.628 2
0.611
In the red box is the final model with the minimum average normalized-BIC

13. Objectives
We describe a SAS Macro for selecting a common set of variables for predicting multiple clinical outcomes using PHREG
The selection method uses a variant of backward elimination based on the average normalized BIC across multiple outcomes
We test our method using the Health and Retirement Study (HRS) data
Compare final predictor set from BIC Combined Backward Elimination with final predictor sets from BIC Backward Elimination by Outcome
Perform simulation study

14. BIC Backward Elimination by Individual Outcome
Ran the BIC backward elimination macro for each individual outcome and selected a final model for each outcome with the minimum BIC
Created a set of predictors with the union of all predictors in the final model of each outcome
Compared it with the set of predictors obtained using combined outcomes

15. BIC Backward Elimination by Individual Outcome (cont.)
Harrell's C-statistic
time to first ADL dependence (9 variables)
time to first IADL difficulty (8 variables)
time to first walking-across-the-room dependence (7 variables)
time to death (16 variables)
Average
Individual models
0.637
0.635
0.636
0.711
Combined model by outcome (14 variables)
0.639
0.642
0.706
0.655 (average 4 models)
Union model by outcome (22 variables)
0.647
0.638
0.648
0.661 (average 4 models)
Combined model is more parsimonious with 14 variables vs variables in union model
C-statistic in the combined model by outcome is very similar to the predictive accuracy of individual and union models
Final combined model has a good balance between parsimony and predictive accuracy

16. Objectives
We describe a SAS Macro for selecting a common set of variables for predicting multiple clinical outcomes using PHREG
The selection method uses a variant of backward elimination based on the average normalized BIC across multiple outcomes
We test our method using the Health and Retirement Study (HRS) data
Compare final predictor set from BIC Combined Backward Elimination with final predictor sets from BIC Backward Elimination by Outcome
Perform simulation study

17. Simulation Study
Created two sets of 100 simulated data sets with the same sample size (N=5,531) and number of initial predictors (i.e. 39)
Generated one set of 100 simulated data sets with high correlation among the outcomes and a second set of simulated data sets with low correlation among the outcomes
Ran the BIC combined backward elimination macro in each of the simulated data sets
Computed the percentage of correct inclusion and the percentage of correct exclusion of predictors in the final simulated models

18. Simulation Study (cont.)
Variable
Percentage Correct Inclusion Uncorrelated outcomes
Percentage Correct Inclusion Correlated outcomes
COGDLRC3G
100
DIABETES
DRIVE 95 89
EDU 78 71
EXERCISE 93 91
HEARTFAILURE 94 96
INCONTINENCE 81
LUNG 73 74
OTHERCLIM3G 87 82
OTHERPUSH 97 92
SMOKING
VOLUNTEER 98
Mean
91.33
89.25
Most of the predictors have a percentage of correct inclusion of 80% or higher. That is, they show up in % of the final combined models using simulated data sets
Percentage of correct exclusion is almost 100% (result not shown in table)

19. Simulation Study (cont.)
Harrell's C-Statistic
No. Variables Final Combined Model
Original data set
0.655 14
Simulated data sets with uncorrelated outcomes (mean [95% CI])
0.647 [ ]
[ ]
Simulated data sets with correlated outcomes (mean [95% CI])
0.646 [ ]
[ ]
Computed mean and 95% Confidence Interval (CI) of the C-statistic and the number of predictors in the final combined models from simulated data sets
C-statistic of the final combined models from the simulated data sets is very similar to the C-statistic of the final combined model in the original data
Means of the number of predictors in the final combined models from simulated data sets are slightly smaller

20. Conclusions
It is possible to select a common set of predictors for multiple clinical outcomes using PROC PHREG and a variant of backward elimination based on the average normalized Bayesian Information Criterion (BIC) across outcomes
Our method provides a straightforward approach without compromising parsimony or predictive accuracy
This method could also be applied to continuous or categorical outcome variables, and other Information Criteria like the AIC can be used

21. Contact Information
Name: Grisell Diaz-Ramirez Company: University of California, San Francisco