Agreement between Methods Karlijn J. van Stralen¹, Kitty J. Jager¹, Carmine Zoccali², and Friedo W. Dekker 1,3 1 ERA–EDTA Registry, Dept. of Medical Informatics, Academic Medical Center, Amsterdam, The Netherlands 2 CNR–IBIM Clinical Epidemiology and Pathophysiology of Renal Diseases and Hypertension, Renal and Transplantation Unit, Ospedali Riuniti, Reggio Cal., Italy 3 Department of Clinical Epidemiology, Leiden University Medical Centre, Leiden, The Netherlands Kidney International: Series on epidemiology

Agreement Situation in which one would like to compare different tests For example GFR estimate by 24 hour creatinine clearance and by inulin clearance Ideal method –Determines whether a relationship is linear –Detect a possible systematic difference –Determine the amount of random difference

Pearsons correlation coefficient Reflects degree of linear relation – -1 perfect negative relation – 0 no correlation – +1 perfect positive relation However, use has several limitations –Type of association –Systematic difference –Range of values

Six different data collections Figures A, B, C & D show different types of associations Figures E and F show a systematic difference. –In E all values are 5 points higher than A (dashed line) –In F all values are multiplied by 2.5 and 10 was subtracted. All pearsons correlation coefficients are 0.80 Shape of the curve and systematic difference

Range of values Entire range of values –High correlation coefficient Narrow range of values –Lower correlation coefficient

Other methods Mean difference + paired t-test –Reveals systematic difference –But – mean difference can be zero if difference is not constant Intraclass correlation coefficient (ICC) –Contain information on correlation and systematic difference –But – sensitive to range of values Regression analysis –Useful for description amount of change –But – no information on size and amount of difference

Bland – Altman plot Difference between the paired measurements (A-B) is plotted against their mean value ([A+B]/2)

Bland - Altman Easy to assess –level of (systematic) difference –the scatter of the values /random error –shows relation between the values and measurement error Limits of agreement –Shows variation in results –Useful for determining interchangeability

Repeated measurements In case of repeated measurements of two methods on the same subject Preferably use all data to compare the two methods. However => do not use the mean –Does result in an accurate estimate of the systematic difference –but underestimation of standard deviation of the differences, and too narrow limits of agreement. Different accurate methods have been developed and described in the following papers –Bland JM, Altman DG: Measuring agreement in method comparison studies. Stat Methods Med Res 8: , 1999 –Myles PS, Cui J: Using the Bland-Altman method to measure agreement with repeated measures. Br J Anaesth 99: , 2007

Calibration Plot of the difference against truth Easy to determine whether a technique should be changed Not useful to determine agreement As these plots show a correlation between the difference and the truth even if there is no difference between the measurement error and the size of the values of the truth

In summary Hard to determine agreement Do not use the correlation coefficient –Sensitive for range of values –Does not reveal a systematic difference –No information on type of association Bland-Altman plot is highly preferred