April 26, 2006 Class 9 Class 9 Tutor Professor: Dra. Cristina Santos Introdução à Medicina Did the quality of diagnostic test accuracy studies in infectious diseases improve in the last twelve years ? Systematic Review

April 26, 2006 Better diagnostic test Faster and better diagnostic Better treatmentBetter prognostic Improvement in patients’ quality of life Introduction

April 26, 2006 STARD To improve the quality of reporting of diagnostic accuracy studies the Standards for Reporting of Diagnostic Accuracy (STARD) was published. (1) The STARD project group developed a checklist of 25 items and a prototypical flow chart to represent the design of the study and the flow of patients through the study. (2) (1) Bossuyt PM et al, de Vet HC: Towards complete and accurate reporting of studies of diagnostic accuracy, the STARD initiative. Clinical Chemistry 2003 (2) Bossuyt PM et al, de Vet HC: Towards complete and accurate reporting of studies of diagnostic accuracy, the STARD initiative. Clinical Chemistry 2003Introduction

April 26, 2006 Aim Evaluate the quality of reporting of diagnostic studies in infectious diseases in high impact factor journals between 1995 and 2006 using the Standards for Reporting of Diagnostic Accuracy (STARD) initiative parameters. Compare the quality of reporting of diagnostic studies in infectious diseases before and after STARD  any improvement?Introduction

April 26, 2006 Query used on PubMed Expressions related to the evaluation of diagnostic tests accuracy (1) Expressions related to infectious diseases 13 journals: - 8 journals – table journals – table 2 Exclusion of review articles and letters to the author (1) Devillé, W.L. et al.: Conducting systematic reviews of diagnostic studies: didactic guidelines. BMC Medical Research Tethodology Jully 2002 Methods

April 26, 2006Methods Rank Abbreviated Journal Title (linked to journal information) Impact Factor 1LANCET INFECT DIS CLIN INFECT DIS AIDS EMERG INFECT DIS ANTIVIR THER J INFECT DIS CURR OPIN INFECT DIS INFECT IMMUN J ANTIMICROB CHEMOTH JAIDS-J ACQ IMM DEF PEDIATR INFECT DIS J CURR HIV RES DIAGN MICR INFEC DIS SEX TRANSM DIS CLIN MICROBIOL INFEC SEX TRANSM INFECT J VIRAL HEPATITIS AIDS RES HUM RETROV INT J ANTIMICROB AG INFECT CONT HOSP EP Smidt et al. Quality of Reporting of Diagnostic Accuracy Studies. Radiology, :350

April 26, 2006 Articles review Years selected: 1995 to 2006 Random selection of 20 articles / year 10 reviewers Each abstract was read by 2 reviewers Apply inclusion / exclusion criteria Disagreement  3 rd reviewer  consensus between 3 reviewersMethods

April 26, 2006 Exclusion criteria 1 – Doesn’t refer to infectious diseases 2 – It’s not an evaluation of diagnostic tests 3 – Other language than English or Portuguese 4 – It’s not an article Note: If the abstract isn’t available, the article should be included.Methods

April 26, 2006Methods Articles evaluation  STARD’s 25 items Each article is evaluated by 2 reviewers Article score = number of STARD items Average score per year

April 26, 2006Methods Agreement between reviewers: In abstract inclusion – PA and kappa In full text evaluation – ICC and Bland and Altman limits Check if the distribution of the items is normal One - WAY ANOVA test to compare the different biennium T-test to compare the quality of pre and post STARD studies

April 26, 2006 Query 2128 studies found Results

April 26, 2006 Articles / year 1995 – – – – – – – – – – – – 138 Random selection of 20 studies per year Application of exclusion criteria 1995 – – – – – – – – – – – – 8Results

April 26, 2006Results New random selection to obtain 10 articles/year 1995 – – – – – – – – – – – – 10 Application of exclusion criteria Total = 111 articles Articles / year

April 26, 2006Results Agreement between the reviewers in abstract inclusion PA = Kappa = 0.77

April 26, 2006 Results Year of publicationMean , , , , , , , , , , , ,70 Total 15,48 Means of the STARD score/year

April 26, 2006 Results Year of publicationMean 95 /96 13,89 97 / 98 14,08 99 /00 17,50 01 / 02 14,45 03 / 04 16,55 05 / 06 15,75 Total 15,48 Means of the STARD score/ biennium

April 26, 2006Results Agreement between the reviewers in full text evaluation ICC = 0,92 Bland and Altman plot

April 26, 2006Results Kolmogorov-Smirnov Test p=0.383 Normal distribution

April 26, 2006 p = Differences in STARD score between years ARE statistically significantResults One-way ANOVA test

April 26, 2006Results Post Hoc Tests Differences between 95/96 and 99/00 ARE statistically significant (p = 0, 027)

April 26, 2006Results Graph of the means of the STARD score/ biennium

April 26, 2006Results Compare means before and after STARD T - test (independent samples) p = 0,072

April 26, 2006Results Presence of STARD items

April 26, 2006 Quality of the studies: medium Mean = Std. Deviation = Discussion improve between 95/96 and 99/00 didn’t improve after 2000 didn’t improve after STARD (2003)

April 26, % or more of the studies analysed didn’t refer: The number of the persons executing the tests if the readers were blind (masked) beginning and ending dates of patients’ recruitment time interval from the index tests to the reference standard adverse events and indeterminate results and how they were handled Discussion

April 26, 2006 Discussion The items 13 and 24 13Methods for calculating test reproducibility, if done 24Measures of test reproducibility, if done These STARD items should NOT be facultative!! If the evaluations followed the STARD statement and measured the reproducibility, they would probably be better.

April 26, 2006 The End