The impact of the STARD checklist on the reporting of studies of diagnostic accuracy in obstetrics and gynecology literature Ana Costa, André Sena, Duarte Pinto, Inês Brandão, João Silva, Lídia Sousa, Luís Barros, Márcia Marinho, Maria Rocha, Pedro Falcão, Rafael Silva, Sara Tapa, Susana Franco. Class 1: Ana Costa, André Sena, Duarte Pinto, Inês Brandão, João Silva, Lídia Sousa, Luís Barros, Márcia Marinho, Maria Rocha, Pedro Falcão, Rafael Silva, Sara Tapa, Susana Franco. Supervisors: Altamiro da Costa Pereira, MD, PhD; Cristina Santos, MD, PhD Introduction to Medicine; Porto Faculty of Medicine 2006

Index 1) Introduction 2) Objective 3) Methods 4) Acknowledgements

Obstetrics and Gynecology are branches of medicine.(*) These areas depend on diagnostic tests.(*) Studies of the new diagnostic tests, and their development, are reported in scientific articles. (**) Introduction (*) - Cunningham, F.Gary. Williams obstetrics, 26ª edição (2005) - Berek, Jonathan S. Novak’s gynecology 12ª edição (1996) (**) - Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studios of diagnostic accuracy: the STARD initiative.Annuals of internal Medicine. 2003; 138:40-44

The reports not always are well made and this may lead to unwanted consequences. (*) STARD was created to guideline the reporting of studies of diagnostic accuracy. (*) Introduction (*) - Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studios of diagnostic accuracy: the STARD initiative.Annuals of internal Medicine. 2003; 138:40-44

To evaluate the impact of the introduction of the STARD checklist on the reporting of diagnostic accuracy, in obstetrics and gynecology literature. Objective

Articles’ search Articles’ selection Selected Articles’ analysis Statistical analyze of the extracted data Methods

Query used in MedLine: (((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All Fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields]) OR ((((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All Fields]) OR "roc and"[All Fields]) OR "roc area"[All Fields]) OR "roc auc"[All Fields]) OR "roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields]) OR ("likelihood ratio"[All Fields])) AND (hum reprod update[Journal] OR Obstet Gynecol[Journal] OR Menopause [Journal] OR Hum Reprod[Journal] OR Obstet Gynecol Surv[Journal] OR Fertil Steril[Journal] OR Am J Obstet Gynecol[Journal]) This query was based on a search strategy in PubMed (MEDLINE) for publications about the evaluation of diagnostic accuracy, suggested by a research article: Devillé, W. L. et. al., Conducting systematic reviews of diagnostic studies: didactic guidelines Articles’ search Methods

Articles’ search

Methods Articles’ selection Creation of 2 groups (with 2 elements each*): - one group read and analyse the abstract of each article published in the other do the same for the those published in 2005 *Note: each element of the group works independently and in the end, if occurs a disagreement about de inclusion of exclusion of an article the entire article is read. Even so, if no agreement is achieved, an expert e called to make the final decision

The article is a report of a study of diagnostic accuracy The diagnostic test is used in obstetrics and/or genecology Was published in 1998 or in 2005 Inclusion criteria: Methods Articles’ selection

The article could not be found in the Internet, the facilities of the School of Medicine or the local libraries The article is written in languages other than English The article is a clinical trial, a letter, an editorial, a practice guideline, a meta-analyse or a review Exclusion criteria: Methods Articles’ selection

STARD checklist Section and topic Item Describe Value Article score TITLE/ABSTRACT/ KEYWORDS 1 The article as a study on diagnostic accuracy (recommend MeSH heading 'sensitivity and specificity') 1 INTRODUCTION 2 The research question(s), such as estimating diagnostic accuracy or comparing accuracy between tests or across participant groups 1 METHODS 1 Participants 3 The study population: the inclusion and exclusion criteria, setting(s) and location(s) where the data were collected 1 4 Participant recruitment: was this based on presenting symptoms, results from previous tests, or the fact that the participants had received the index test(s) or the reference standard? 1 5 Participant sampling: was this a consecutive series of patients defined by selection criteria in (3) and (4)? If not specify how patients were further selected. 1 6 Data collection: were the participants identified and data collected before the index test(s) and reference standards were performed (prospective study) or after (retrospective study)? 1 Reference standard 7 The reference standard and its rationale 1 Test methods 8 Technical specification of material and methods involved including how and when measurements were taken, and/or cite references for index test(s) and reference standard 0,50,5 9 Definition and rationale for the units, cutoffs and/or categories of the results of the index test(s) and the reference standard 0,50,5 10 The number, training and expertise of the persons (a) executing and (b) reading the index test(s) and the reference standard 0,50,5 11 Whether or not the reader(s) of the index test(s) and reference standard were blind (masked) to the results of the other test(s) and describe any information available to them 0,50,5 Statistical methods 12 Methods for calculating measures of diagnostic accuracy or making comparisons, and the statistical methods used to quantify uncertainty (e.g. 95% confidence intervals) 1 13 Methods for calculating test reproducibility, if done 0,50,5 Methods Selected Articles’ analysis

Methods Selected Articles’ analysis STARD checklist Section and topic ItemDescribeValue Article score RESULTS Participants 14 When study was done, including beginning and ending dates of recruitment 1 15 Clinical and demographic characteristics (e.g. age, sex, spectrum of presenting symptom(s), comorbidity, current treatment(s), recruitment center) 1 16 How many participants satisfying the criteria for inclusion did or did not undergo the index test and/or the reference standard; describe why participants failed to receive either test (a flow diagram is strongly recommended) 1 Reference standard 17 Time interval and any treatment administered between index and reference standard 1 18 Distribution of severity of disease (define criteria) in those with the target condition; describe other diagnoses in participants without the target condition 1 Test results 19 A cross tabulation of the results of the index test(s) by the results of the reference standard; for continuous results, the distribution of the test results by the results of the reference standard 1 20 Indeterminate results, missing responses and outliers of index test(s) stratified by reference standard result and how they were handled 1 21 Adverse events of index test(s) and reference standard 1 Estimation 22 Estimates of diagnostic accuracy and measures of statistical uncertainty (e.g. 95% confidence intervals) 1 23 Estimates of variability of diagnostic accuracy between subgroups of participants, readers or centers, if done 1 24 Measures of test reproducibility, if done 0,50,5 DISCUSSION 25 The clinical applicability of the study findings 1 Bossuyt, P. M. et. al., Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative, Annals of Internal Medicine Vol 138 – nº1, 7 January 2003

Methods Selected Articles’ analysis 3 groups (  ): - G1: Title, Abstract, Keywords, Introduction - G2: Methods - G3: Results, Discussion 1 group (α): - Makes the final decision when occur disagreements within  groups in the classification of the article - In the end, analyses 1/3 of the articles, randomly selected

Final score of each article. Mean score of the articles for each year (1998 and 2005) Comparison of the 2 means Interobserver variability The status of each STARD item in each year Methods Statistical analyze of the extracted data Points to analyse:

Creation of a data base, in SPSS Determination of the scores of each article Execution of an Inter-class correlation coefficient and the limits of Bland and Altaman Determination of mean, range, and standard deviation of each item of the STARD, for each year Determination of mean score of the articles, for each year Execution of an unpaired two-tailed t-test for independent samples (significance level 0,05) Methods Statistical analyze of the extracted data How ?

Results ( Articles’ search )

Planning

Professor Altamiro da Costa Pereira Professor Cristina Santos Acknowledgments