Final Presentation Group 2 Porto, 2 nd May 2007 Medical Faculty of Oporto

1.Introduction 2.Methods 3.Results 4.Discussion and Conclusions 5.Acknowledgements

As an extremely dynamic field, medicine evolves each day, integrating new ideas and concepts which come from all over the world Gastroenterology and hepatology are medical specializations concerned with the diagnosis and treatment of diseases of the digestive tract and organs Diagnostic tests play a crucial role in the accurate diagnose of diseases

The number of existing diagnostic tests has grown over the past few years 1 In January 2003, the guide for reporting studies of diagnostic accuracy Standards for Reporting of Diagnostic Accuracy (STARD) was published 1 The objective of the STARD initiative is to improve the quality of reporting of studies of diagnostic accuracy 1 1- Bossuyt, P; Reitsma, J; Bruns, D; Gatsonis, C; Glasziou, P; Irwig, L; Lijmer, J; Moher, D; Rennie, D; de Vet, H, Towards complete and accurate reporting o studies of diagnostic accuracy: the STARD initiative; Annals of internal medicine; Volume 138, 2003.

Section and TopicItem #On page # TITLE/ABSTRACT/KEYWORDS1 Identify the article as a study of diagnostic accuracy (recommend MeSH heading `sensitivity and specificity'). INTRODUCTION2 State the research questions or study aims, such as estimating diagnostic accuracy or comparing accuracy between tests or across participant groups. METHODS Participants3 Describe the study population: The inclusion and exclusion criteria, setting and locations where the data were collected. 4 Describe participant recruitment: Was recruitment based on presenting symptoms, results from previous tests, or the fact that the participants had received the index tests or the reference standard? 5 Describe participant sampling: Was the study population a consecutive series of participants defined by the selection criteria in items 3 and 4? If not, specify how participants were further selected. 6 Describe data collection: Was data collection planned before the index test and reference standard were performed (prospective study) or after (retrospective study)? Test methods7Describe the reference standard and its rationale. 8 Describe technical specifications of material and methods involved including how and when measurements were taken, and/or cite references for index tests and reference standard. 9 Describe definition of and rationale for the units, cutoffs and/or categories of the results of the index tests and the reference standard. 10 Describe the number, training and expertise of the persons executing and reading the index tests and the reference standard. 11 Describe whether or not the readers of the index tests and reference standard were blind (masked) to the results of the other test and describe any other clinical information available to the readers. Statistical methods12 Describe methods for calculating or comparing measures of diagnostic accuracy, and the statistical methods used to quantify uncertainty (e.g. 95% confidence intervals). 13Describe methods for calculating test reproducibility, if done. RESULTS Participants14Report when study was done, including beginning and ending dates of recruitment. 15 Report clinical and demographic characteristics of the study population (e.g. age, sex, spectrum of presenting symptoms, comorbidity, current treatments, recruitment centers). 16 Report the number of participants satisfying the criteria for inclusion that did or did not undergo the index tests and/or the reference standard; describe why participants failed to receive either test (a flow diagram is strongly recommended). Test results17 Report time interval from the index tests to the reference standard, and any treatment administered between. 18 Report distribution of severity of disease (define criteria) in those with the target condition; other diagnoses in participants without the target condition. 19 Report a cross tabulation of the results of the index tests (including indeterminate and missing results) by the results of the reference standard; for continuous results, the distribution of the test results by the results of the reference standard. 20Report any adverse events from performing the index tests or the reference standard. Estimates21 Report estimates of diagnostic accuracy and measures of statistical uncertainty (e.g. 95% confidence intervals). 22 Report how indeterminate results, missing responses and outliers of the index tests were handled. 23 Report estimates of variability of diagnostic accuracy between subgroups of participants, readers or centers, if done. 24Report estimates of test reproducibility, if done. DISCUSSION25Discuss the clinical applicability of the study findings. Bossuyt, P; Reitsma, J; Bruns, D; Gatsonis, C; Glasziou, P; Irwig, L; Lijmer, J; Moher, D; Rennie, D; de Vet, H, Towards complete and accurate reporting o studies of diagnostic accuracy: the STARD initiative; Annals of internal medicine; Volume 138, 2003.

Objectives Evaluate the quality of the articles which evaluate diagnostic tests in Gastroenterology and Hepatology on journals with the highest impact factor in the past 12 years based on the STARD checklist, and unveil if there was evolution in this type of studies Assess if the STARD initiative had influence in the articles or not 1 - Bossuyt, P; Reitsma, J; Bruns, D; Gatsonis, C; Glasziou, P; Irwig, L; Lijmer, J; Moher, D; Rennie, D; de Vet, H, Towards complete and accurate reporting o studies of diagnostic accuracy: the STARD initiative; Annals of internal medicine; Volume 138, 2003.

1.Introduction 2.Methods 3.Results 4.Discussion and Conclusions 5.Acknowledgements

1.Article search 2.Selection of abstracts 3.Article evaluation

Initial Query for studies of diagnostic accuracy Definition of inclusion parameters  Restriction to the five journals of gastroenterology and hepatology with the highest impact factor  Restriction to diagnostic tests in humans

Initial Query (((((((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields)) OR (((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All fields]) OR "roc and"[All fields]) OR "roc area"[All fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields])) OR "likelihood ratio"[All Fields]) Ref:Dévillé WL, Buntins F, Bouter LM, Montori, VM, de Vet H, Windt DAWM, Bezemer PD. Conducting systematic reviews of diagnostic studies: didactic guidelines, BMC medical research methodology 2002, 2:9

Initial Query (((((((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields)) OR (((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All fields]) OR "roc and"[All fields]) OR "roc area"[All fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields])) OR "likelihood ratio"[All Fields]) AND (Humans[MeSH])

Year 1994 Articles Found – 174 Query parameters: (…) 1994 [Publication Date] AND (GASTROENTEROLOGY [Journal] OR HEPATOLOGY [Journal] OR GUT [Journal] OR GASTROINTEST ENDOSC [Journal] OR J HEPATOL [Journal])

1994 – – – – – – – – – – – – 299 Journals with highest occurence: Gastroenterol Hepatol Gut

1.Article search 2.Selection of abstracts 3.Article evaluation

Non-excluded and non-included abstracts were listed along with reasons of non-inclusion and added to the data base on SPSS

1.Article search 2.Selection of abstracts 3.Article evaluation

1.Introduction 2.Methods 3.Results 4.Discussion and Conclusions 5.Acknowledgements

Does not assess diagnostic tests Missing abstract Other Reasons for non-inclusion 102 abstracts not excluded from 300 found Comparison between excluded and included articles

Agreement between abstract reviewers Rate of agreement between reviewers – 84,3%

Does not assess diagnostic tests Missing article Exclusion reasons (%) 78 articles included from 102 abstracts not excluded Comparison between excluded and included articles

Agreement between article reviewers 1 - Smidt, N.; Rutjes, A.; van der Windt, D.; Ostelo, R.; Bossuyt, P; Reitsma, J.; Bouter, L; de Vet, H; Reproducibility of the STARD checklist: an instrument to assess the quality of reporting of diagnostic accuracy studies; BMC medical research methodology; 2006 Obtained ICC value is in accordance with ICC to be expected by Smidt, N. Expected ICC = 0,79 [95% CI: 0,62 to 0,89] 1

Evolution of the article’s score by mean

Evolution of the article’s score by median

Items with highest absence (i13, i24) refer to reproducibility issues Individual STARD items absence percentage

1.Introduction 2.Methods 3.Results 4.Discussion and Conclusions 5.Acknowledgements

STARD did not had a significant effect in the article’s quality Insufficient time span between the STARD publishing (2003) and the current date The small evolution in the article’s quality is likely to be a consequence of other factors, not taken into account in this study This study should be repeated in a wider time span

The quality of the this kind of articles in the journals with the highest impact factors is not as positive as expected The credibility of these articles may be compromised The results presented may be misleading

The STARD items referring to reproducibility issues are highly absent The studies might be difficult to reproduce, which may lead to difficulties in the repeating of the studies by other researchers and therefore may impede verification of results There is the possibility of the diagnostic tests assessed to be inaccurate

1.Introduction 2.Methods 3.Results 4.Discussion and Conclusions 5.Acknowledgements

Cristina Santos, MD Altamiro da Costa Pereira, PhD MD

Porto, 2 nd May 2007 Medical Faculty of Oporto Adriana Meneses Ana Fragoeiro André Carvalho Daniela Machado Eduardo Vilela Inês Gonçalves João Sarmento Luís Santos Manuel Pinto Pedro Alves Sara Ferreira Tiago Taveira Orientation Cristina Santos, MD Supervisor Altamiro da Costa Pereira, PhD MD Authors