A SAS Macro for Displaying Institutional Risk-adjusted Performance: Forest vs. Funnel Plots Zhongmin Li and Geeta Mahendra University of California Davis Medical Center May 20, 2014

Overview Introduction Data preparation SAS Macro for Forest vs. Funnel plots Discussion

Introduction Both forest plot and funnel plot were initially developed for presenting results of meta-analysis Forest plot: an intuitive, convenient and easily understood display of mean values and their confidence intervals for comparisons Funnel plots: no spurious ranking; the eye is naturally drawn to important points that lie outside the funnel

Data Preparation 20 observations; one per institution Variables: Institutional ID, total number of cases admitted, number of adverse events, observed rate of adverse events, Expeted rate of adverse event, Risk adjusted event rate and the lower and upper limits of 95% confidence interval of the risk adjusted event rate.

Sample Data

Forest Plot (1) %MACRO ForestPlot (Datain, PopStateRate); … DATA Anno; LENGTH function style color $8; RETAIN xsys ysys '2' when 'a'; SET &Datain; **Drawing horizontal line for CI of risk-adjusted rate; FUNCTION='move'; xsys='2'; ysys='2'; yc=hospname; x=LL_RAMR; color='black'; output; FUNCTION='draw'; x=UL_RAMR; color='black'; size=1; output;

Forest Plot (2) **Drawing tick lines at both ends of the lower and the upper limits Function='move';xsys='2'; ysys='2';yc=hospname; x=ll_RAMR; color='black'; output; Function='draw';x=ll_RAMR; ysys='9'; y=+1; size=1; output; Function='draw';x=ll_RAMR; y=-2; size=1;output; Function='move';xsys='2'; ysys='2'; yc=hospname; x=Ul_RAMR; color='black'; output; Function='draw';x=Ul_RAMR; ysys='9'; y=+1; size=1; output; Function='draw';x=Ul_RAMR; y=-2; size=1; output; RUN;

Forest Plot (3) **Create Y axis values in ascending order of the risk adjusted event rate for the graph. PROC SQL is used to create a macro variable HOSP which is then used to order the Y axis labels; PROC SQL noprint; select quote(hospname) into :hosp separated by ' ' from &datain; Quit; Axis1 label=none minor=none offset=(2,2) order=(&hosp); Axis2 order=(0 to 18 by 2) label=('Risk Adjusted Event Rate') minor=none; Symbol1 interpol=none color=black value=dot height=1.5;

Forest Plot (4) **Use GPLOT to create the forest plot by plotting hospital name on the Y axis and risk adjusted event rate on the X axis. A vertical reference line for the state population rate is added using the HREF option. PROC GPLOT DATA=&datain; Plot Hospname*adjrate / annotate=anno nolegend Vaxis=axis1 Haxis=axis2 Href = &PopStateRate LHref = 2; Run;   %MEND ForestPlot;

Forest Plot: Results

Funnel Plot (1) %MACRO FunnelPlot(Datain, MeanProcRate, Sampsize); … **Use SAS IML to generate 95% and 99.8% confidence bands around the average of mean risk adjusted event rate. PROC iml; USE Rpt; READ all var {hospname Voln nobs adjrate se}; CLOSE rpt;

Funnel Plot (2) **Create IML module FunnelStat for matrixes. START FunnelStat(x); mean = x[:,]; countn = j(1, ncol(x)); do i = 1 to ncol(x); countn[i] = sum(x[,i]^=.); end; FunnelStat = (x-mean)[##,] / (countn-1); RETURN ( FunnelStat ); FINISH;

Funnel Plot (3) **Calculate overall mean, variance and the lower and upper limits for 95% and 99.8% confidence bands for mean risk adjusted event rate; y = adjrate[:]; s = sqrt(FunnelStat(adjrate)); n = T( do(1, &sampsize, 1) ); p = {0.001 0.025 0.975 0.999}; /** lower/upper limits **/ z = quantile("normal", p); limits = y + s / sqrt(n) * z;

Funnel Plot (4) **Create a dataset which has all the variables needed to create the funnel plot by combining confidence limits with the rest of the data. d = n || limits; VARNAMES = {"N" "L998" "L95" "U95" "U998" }; CREATE Bounds from d[c=varNames]; APPEND from d; CLOSE Bounds; QUIT; **If statement is used to label only the outliers. DATA plotdata; SET Temp Bounds; outlier=Hospname; if hospname not in ('Unit#8','Unit#15') then outlier=' ';

Funnel Plot (5) **Using SGPLOT procedure. PROC SGPLOT DATA=plotdata ; SCATTER x=voln y=adjrate /datalabel=Outlier ; REFLINE &MeanProcRate / axis=y; BAND x=N lower=l95 upper=U95 / nofill name="CI95" LegendLabel="95% limits" lineattrs=(color=gray); … XAXIS label="Procedure Volume Category" fitpolicy=rotate grid values=(1 to 10 by 1) integer ; YAXIS label="Mean Risk-adjusted Event Rate"; KEYLEGEND "CI998" "CI95" / location=inside position=TopRight across=1; %MEND FunnelPlot;

Funnel Plot: Results

Discussion Both Forest and Funnel plots can be used to rate institutions based on their performance. Forest plots are used more for ranking purposes; Funnel plots are a simple way to compare group means to overall average without ranking Funnel plots use the point estimates which could be caused by a chance. Primary audiences for funnel plots are more likely to be researchers and analysts than senior decision makers.

Comments? Please contact: Zhongmin Li Zhongmin.Li@ucdmc.ucdavis.edu Geeta Mahendra Geeta.Mahendra@ucdmc.ucdavis.edu