Examining model stability, an example proc logistic data=s5238.chd5238 outest=betas; model chd10yr(event="1")=age male sbp chol diab currsmok/ selection=forward; run; proc print data=betas;run;

Select 100 bootstrap samples %bootsamp(indat=s5238.chd5238,outdat=outboot,reps=100,seed=3759543);

Re-do analysis for each bootstrap sample proc logistic data=outboot outest=betas noprint; by replicate; model chd10yr(event="1")=age male sbp chol diab currsmok/ selection=forward; run; proc print data=betas;run;

A simple summary. proc means data=betas; var age male sbp chol diab currsmok; run;

Another summary – create a string that is six characters long Another summary – create a string that is six characters long. Each character is 0 or 1. The first character is 1 if the first variable is in the model, 0 if it is not. The same for the other two characters.

%macro zeroone(indat=,outdat=models,newvar=model,vars=); %let numvars=%sysfunc(countw(&vars)); data &outdat; length &newvar $ &numvars..; set &indat; array t{&numvars} &vars; do i=1 to &numvars; substr(&newvar,i,1)=put((t{i} ne .),1.); end; run; %mend;

%zeroone(indat=betas,outdat=models,newvar=model,vars= age male sbp chol diab currsmok); proc freq data=models; table model; run;

A much more complex model A much more complex model. Lots of interaction terms, no hierarchical model restrictions. proc logistic data=s5238.chd5238 outest=betas descending ; model chd10yr=age | male | chol | sbp | bmi | diab | currsmok / selection=forward slentry=.1 hierarchy=none;/*single is default*/ run;

Do the complex model on the bootstrap samples. proc logistic data=outboot outest=betas descending noprint; by replicate; model chd10yr=age | male | chol | sbp | bmi | diab | currsmok / selection=forward slentry=.1 hierarchy=none;/*single is default*/ run;

The problem -- Lots of variable names. %zeroone(indat=betas,outdat=models,newvar=model,vars= age male sbp chol diab currsmok); proc contents data=betas; run;

Use SQL to put variables names into a macro variable proc sql; select name into :varnames separated by " " from dictionary.columns where libname="WORK" and memname="BETAS" and type="num" and lowcase(name) ne "replicate" and lowcase(name) ne "intercept" and name ne "_LNLIKE_" ; quit; %let numvars=%sysfunc(countw("&varnames")); %put &numvars; %put &varnames;

Now use the zeroone macro %zeroone(indat=betas,outdat=models1,newvar=model,vars=&varnames); proc sql; create table nummodels as select distinct model from models1 ; select count(*) "Number of distinct models" from nummodels quit;