A SAS Macro to Calculate the C-statistic Bill O’Brien BCBSMA SAS Users Group March 10, 2015

C-statistic measures discrimination A key component in the assessment of risk algorithm performance is its ability to distinguish subjects who will develop an event (“cases”) from those who will not (“controls”). This concept, known as discrimination, has been well studied and quantified for binary outcomes using measures such as the estimated area under the Receiver Operating Characteristics (ROC) curve (AUC), which is also referred to as a “C-statistic” (Uno 2011)

data Admissions; length patientID $ 3 predicted 8 actual 3; do k=1 to 100; patientID=put(k,z3.); actual=(rand("uniform") gt.8); predicted=rand("normal",0.2,0.08)+(actual*0.07); output; end; run; Sample Dataset patientIDpredictedactual % % % % %0 PriorPost

Discrimination Slope proc tabulate data=admissions; class actual; var predicted; tables predicted*(n*f=2.0 mean*f=percent8.1), actual; run; actual 01 predictedN8317 Mean20.7%27.0%

Algorithm 1.Cartesian join all events to all non-events 2.Assign row a value of 1 if Pr(event) > Pr(non-event); 0 if not; 0.5 for tie 3.Take the average

Calculating the c-statistic %macro cstatistic(dsn,predvar,event); proc sql; select (sum(cVal))/count format=8.2 as c into: cStat from (select case when t1.LL>t2.LL then 1 when t1.LL=t2.LL then 0.5 else 0 end as cVal,count(*) as count from (select &predvar as LL from &dsn where &event=1) as t1, (select &predvar as LL from &dsn where &event=0) as t2 ); quit; %mend cstatistic; %cstatistic(admissions,predicted,actual); Cartesian join: All events to all non-events +1 if the model assigned higher p to event vs. non- event % of time model discriminated correctly

Result c 0.72 There is a 0.72 probability of the model assigning a higher predicted probability to a randomly selected event case, compared with a randomly selected non-event case = no better than chance 0.60 = poor 0.70 = reasonable 0.80 = strong (Hosner & Lemeshow 2000)

Further Reading