1. SRDR Quarterly Training Brown Evidence-based Practice Center Brown University June 20 th, 2014 1:00pm-2:00pm Entering Data Retrospectively into SRDR The Systematic Review Data Repository (SRDR™) was developed by the Brown Evidence-based Practice Center (EPC), Boston, Massachusetts, under contract with the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. HHSA 290-2007-10055-I & HHSA 290-2012-1200012-I ).

2. 6/20/2014http://srdr.ahrq.gov2  To provide guidance on how to efficiently enter data retrospectively into SRDR Presentation objective

3. 6/20/2014http://srdr.ahrq.gov3 Presentation outline The definition of prospective and retrospective data entry into SRDR The benefits and challenges of entering data retrospectively into SRDR Method 1: using the SRDR interface Method 2: using SRDR exported extraction form template(s) Method 3: using source file(s) directly imported into SRDR Future directions

4. 6/20/2014http://srdr.ahrq.gov4 The definition of prospective and retrospective data entry into SRDR  Prospective data entry into SRDR:  Data used to write a report is entered into SRDR as it is being extracted from eligible studies  Entry of data into SRDR is incorporated into the workflow of the report  Retrospective data entry into SRDR:  Data used to write a report is entered into SRDR after it has already been extracted from eligible studies and saved in source file(s)

5. 6/20/2014http://srdr.ahrq.gov5 Presentation outline The definition of prospective and retrospective data entry into SRDR The benefits and challenges of entering data retrospectively into SRDR Method 1: using the SRDR interface Method 2: using SRDR exported extraction form template(s) Method 3: using source file(s) directly imported into SRDR Future directions

6. 6/20/2014http://srdr.ahrq.gov6 The benefits of entering data retrospectively into SRDR  Data entered into SRDR is saved in a structured computable format with options to export to Microsoft Excel or CSV files using SRDR Data Export tools  Data published to SRDR’s published projects page can be viewed and/or downloaded by the public and other SRDR users improving the transparency of your report.  Data published to SRDR’s published projects page can be commented on by SRDR users with project contributor or public commentator accounts, adding to the reliability of your data  Data published to SRDR’s published projects page can be easily updated with new study records or information. In addition, updated data can be published as a separate project.

7. 6/20/2014http://srdr.ahrq.gov7 The challenges of entering data retrospectively into SRDR  Before data can be retrospectively entered into SRDR, data fields on the extraction forms used in the report must be mapped to SRDR extraction form templates which utilize a tabbed structure corresponding to different types of data included in the report.  Data entered retrospectively into SRDR must be transferred, from the source file(s) it was saved in, into the data fields of SRDR extraction form templates for each eligible study in the report  The retrospective data entry process should include at least one member of the original team that wrote the report to facilitate with judgment calls concerning the data

8. 6/20/2014http://srdr.ahrq.gov8 Presentation outline The definition of prospective and retrospective data entry into SRDR The benefits and challenges of entering data retrospectively into SRDR Method 1: using the SRDR interface Method 2: using SRDR exported extraction form template(s) Method 3: using source file(s) directly imported into SRDR Future directions

9. 6/20/2014http://srdr.ahrq.gov9  The method:  Data is entered retrospectively into SRDR, one study at a time, using the SRDR interface  Data is manually entered by EPC team members into SRDR extraction form templates for each eligible study  Data can be transferred from any source file such as a Microsoft Word document, Microsoft Excel spreadsheet, PDF, or export file from another available extraction tool Method 1: using the SRDR interface

10. 6/20/2014http://srdr.ahrq.gov10  Materials from the report required to use this method:  Extraction form templates  List of eligible studies including publication information and which key questions each study is assigned to  Extracted study data  Characteristics of the report suitable for this method:  Report size: small, including only a limited number of eligible studies  Extraction form size: Limited number of datafields Method 1 - requirements

11. 6/20/2014http://srdr.ahrq.gov11 Enter extracted study data from the report into the project Publish the project to SRDR’s published projects page Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key question(s) Initiate a project in SRDR 1 5 6 4 3 2 Method 1- steps Enter extracted study data from the report into the studies added to the project Publish the project to SRDR’s published projects page Add eligible studies included in the report to the SRDR project Create an SRDR extraction form for each extraction form used in the report Input general project information and key question(s) from the report Initiate a project in SRDR 1 5 6 4 3 2

12. 6/20/2014http://srdr.ahrq.gov12  Title: Imaging Techniques for Treatment Evaluation for Metastatic Breast Cancer (Pacific Northwest EPC)  Technical brief with 17 eligible studies  Microsoft Word document source files Method 1- example

13. 6/20/2014http://srdr.ahrq.gov13 Enter extracted study data from the report into the project Publish the project to SRDR’s published projects page Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key question(s) Initiate a project in SRDR 1 5 6 4 3 2 Method 1: using the SRDR interface

14. 6/20/2014http://srdr.ahrq.gov14 Enter extracted study data from the report into the project Publish the project to SRDR’s published projects page Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key question(s) Initiate a project in SRDR 1 5 6 4 3 2 Method 1: using the SRDR interface

15. 6/20/2014http://srdr.ahrq.gov15 Enter extracted study data from the report into the project Publish the project to SRDR’s published projects page Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key question(s) Initiate a project in SRDR 1 5 6 4 3 2 Method 1: using the SRDR interface

16. 6/20/2014http://srdr.ahrq.gov16 Enter extracted study data from the report into the project Publish the project to SRDR’s published projects page Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key question(s) Initiate a project in SRDR 1 5 6 4 3 2 Method 1: using the SRDR interface

17. 6/20/2014http://srdr.ahrq.gov17 Enter extracted study data from the report into the project Publish the project to SRDR’s published projects page Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key question(s) Initiate a project in SRDR 1 5 6 4 3 2 Method 1: using the SRDR interface

18. 6/20/2014http://srdr.ahrq.gov18 Enter extracted study data from the report into the project Publish the project to SRDR’s published projects page Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key question(s) Initiate a project in SRDR 1 5 6 4 3 2 Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Method 1: using the SRDR interface

19. 6/20/2014http://srdr.ahrq.gov19 Enter extracted study data from the report into the project Publish the project to SRDR’s published projects page Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key question(s) Initiate a project in SRDR 1 5 6 4 3 2 Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Method 1: using the SRDR interface

20. 6/20/2014http://srdr.ahrq.gov20 Enter extracted study data from the report into the project Publish the project to SRDR’s published projects page Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key question(s) Initiate a project in SRDR 1 5 6 4 3 2 Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Method 1: using the SRDR interface

21. 6/20/2014http://srdr.ahrq.gov21 Enter extracted study data from the report into the project Publish the project to SRDR’s published projects page Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key question(s) Initiate a project in SRDR 1 5 6 4 3 2 Method 1: using the SRDR interface

22. 6/20/2014http://srdr.ahrq.gov22 Publish the project to SRDR’s published projects page Enter extracted study data from the report into the project Publish the project to SRDR’s published projects page Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 5 6 4 3 2  The method:  Data for each study in the report manually entered into SRDR by an EPC team member  Data entered into SRDR can be transferred from any source file such as a Microsoft Word document, Microsoft Excel spreadsheet, PDF, or export file from another available extraction tool  Materials:  Source files containing extracted data values  Extraction form templates  List of eligible studies including which key questions each study is assigned to  Characteristics:  Extraction forms with a limited number of datafields  Small report size, including a limited number of eligible studies Method 1 - summary

23. 6/20/2014http://srdr.ahrq.gov23 Presentation outline The definition of prospective and retrospective data entry into SRDR The benefits and challenges related to entering data, retrospectively, into SRDR Method 1: using the SRDR interface Method 2: using SRDR exported extraction form template(s) Method 3: using source file(s) directly imported into SRDR Future directions

24. 6/20/2014http://srdr.ahrq.gov24 Method 2: using SRDR exported extraction form template(s)  The method:  Data is entered retrospectively into SRDR for multiple studies at once using SRDR exported extraction form template(s)  Data for each study in the report is manually entered into SRDR exported extraction form template(s) by a member of an EPC (preferably by a member of the EPC that generated the report)  Data entered into SRDR using this method should be transferred from source files saved as Microsoft Excel spreadsheets

25. 6/20/2014http://srdr.ahrq.gov25  Materials from the report required to use this method:  Extraction form templates  List of eligible studies including publication information and which key questions each study is assigned to  Extracted study data  Characteristics of the report suitable for this method:  Report size: large, including any number of eligible studies  Extraction form size: Any number of datafields Method 2 - requirements

26. 6/20/2014http://srdr.ahrq.gov26 Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7 Method 2 - steps Export SRDR extraction form template(s) as Microsoft Excel spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form for each extraction form used in the report Input general project information and key questions from the report Initiate a project in SRDR Enter extracted study data from the report into SRDR exported extraction form template(s) Submit SRDR exported extraction form template(s) filled with extracted study data to SRDR team to import into the appropriate data fields on SRDR extraction forms in the project Publish the project to SRDR’s published projects page 1 4 3 2 6 8 5 7

27. 6/20/2014http://srdr.ahrq.gov27  Title: First- and Second- Generation Antipsychotics for Children and Young Adults (Alberta EPC)  Systematic review with 81 eligible studies  Microsoft Excel source files Method 2 - example

28. 6/20/2014http://srdr.ahrq.gov28 Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7 Method 2: using SRDR exported extraction form templates

29. 6/20/2014http://srdr.ahrq.gov29 Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7 Method 2: using SRDR exported extraction form templates

30. 6/20/2014http://srdr.ahrq.gov30 Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7 Method 2: using SRDR exported extraction form templates

31. 6/20/2014http://srdr.ahrq.gov31 Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7 Method 2: using SRDR exported extraction form templates

32. 6/20/2014http://srdr.ahrq.gov32 Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7 Method 2: using SRDR exported extraction form templates

33. 6/20/2014http://srdr.ahrq.gov33 Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7 Method 2: using SRDR exported extraction form templates

34. 6/20/2014http://srdr.ahrq.gov34 Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7 Method 2: using SRDR exported extraction form templates

35. 6/20/2014http://srdr.ahrq.gov35 Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7 Method 2: using SRDR exported extraction form templates

36. 6/20/2014http://srdr.ahrq.gov36 Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7 Method 2: using SRDR exported extraction form templates

37. 6/20/2014http://srdr.ahrq.gov37 Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7 Method 2: using SRDR exported extraction form templates

38. 6/20/2014http://srdr.ahrq.gov38 Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7 Method 2: using SRDR exported extraction form templates

39. 6/20/2014http://srdr.ahrq.gov39  The method  Data entered retrospectively into SRDR by EPC team member using SRDR exported extraction form template(s)  Data transferred from source files saved as Microsoft Excel spreadsheets  Requirements  Extraction form templates  List of eligible studies including which key questions each study is assigned to  Source files containing extracted study dataa  Characteristics:  Extraction forms with any number of datafields  Medium-sized report, including a limited number of eligible studies Method 2 - summary Export SRDR extraction form template(s) as spreadsheet Add eligible studies included in the report to the SRDR project Create an SRDR extraction form Input general project information and key questions Initiate a project in SRDR 1 4 3 2 Enter extracted study data from the report into template(s) Submit template(s) to SRDR team for import into the project Publish the project to SRDR’s published projects page 6 8 5 7

40. 6/20/2014http://srdr.ahrq.gov40 Presentation outline The definition of prospective and retrospective data entry into SRDR The benefits and challenges related to entering data, retrospectively, into SRDR Method 1: using the SRDR interface Method 2: using SRDR exported extraction form template(s) Method 3: using source file(s) directly imported into SRDR Future directions

41. 6/20/2014http://srdr.ahrq.gov41  The method:  Data is entered retrospectively into SRDR for multiple studies at once using source file(s)  Data saved in Microsoft Excel source file(s) for each study in the report is directly imported into SRDR by the SRDR team  Data that is entered into SRDR using this method, will only be populated on the Design tabs of the SRDR extraction forms for each study in the report Method 3: using source file(s) to directly import extracted study data into SRDR

42. 6/20/2014http://srdr.ahrq.gov42  Materials from the report required to use this method:  Microsoft Excel source files containing extracted study data  Each source file must be formatted such that the extracted study data for each study occupies a single row and column headers correspond to separate datafields on the extraction form that was used to extract the data  Each file must also have columns for “Key Question”, “PMID”, “Internal ID”, “Author”, and “Year”  It is preferred that arms and outcome spreadsheets be included with each source file containing arms and outcomes data  Characteristics of the report suitable for this method:  Report size: large, including any number of eligible studies  Extraction form size: Any number of datafields Method 3 - Requirements

43. 6/20/2014http://srdr.ahrq.gov43 Publish the project to SRDR’s published projects page Submit source file(s) to SRDR team for import into the project Prepare source file(s) for import Input general project information and key questions Initiate a project in SRDR 1 5 4 3 2 Publish the project to SRDR’s published projects page Submit source file(s) to the SRDR team to import into the appropriate data fields on SRDR extraction forms in the project Prepare source file(s) from the report to be directly imported into SRDR Input general project information and key question(s) from the report Initiate a project in SRDR 1 5 4 3 2 Method 3 - steps

44. 6/20/2014http://srdr.ahrq.gov44  Title: Imaging Techniques for Treatment Evaluation for Metastatic Breast Cancer (Pacific Northwest EPC)  Systematic review with 281 eligible studies  Extracted study data saved in Microsoft Excel source files Method 3 - example

45. 6/20/2014http://srdr.ahrq.gov45 Publish the project to SRDR’s published projects page Submit source file(s) to SRDR team for import into the project Prepare source file(s) for import Input general project information and key questions Initiate a project in SRDR 1 5 4 3 2 Method 3: using source file(s) to directly import extracted study data into SRDR

46. 6/20/2014http://srdr.ahrq.gov46 Publish the project to SRDR’s published projects page Submit source file(s) to SRDR team for import into the project Prepare source file(s) for import Input general project information and key questions Initiate a project in SRDR 1 5 4 3 2 Method 3: using source file(s) to directly import extracted study data into SRDR

47. 6/20/2014http://srdr.ahrq.gov47 Publish the project to SRDR’s published projects page Submit source file(s) to SRDR team for import into the project Prepare source file(s) for import Input general project information and key questions Initiate a project in SRDR 1 5 4 3 2 Method 3: using source file(s) to directly import extracted study data into SRDR

48. 6/20/2014http://srdr.ahrq.gov48 Publish the project to SRDR’s published projects page Submit source file(s) to SRDR team for import into the project Prepare source file(s) for import Input general project information and key questions Initiate a project in SRDR 1 5 4 3 2 Original Modified and ready for import Method 3: using source file(s) to directly import extracted study data into SRDR

49. 6/20/2014http://srdr.ahrq.gov49 Publish the project to SRDR’s published projects page Submit source file(s) to SRDR team for import into the project Prepare source file(s) for import Input general project information and key questions Initiate a project in SRDR 1 5 4 3 2 Method 3: using source file(s) to directly import extracted study data into SRDR

50. 6/20/2014http://srdr.ahrq.gov50 Publish the project to SRDR’s published projects page Submit source file(s) to SRDR team for import into the project Prepare source file(s) for import Input general project information and key questions Initiate a project in SRDR 1 5 4 3 2 Method 3: using source file(s) to directly import extracted study data into SRDR

51. 6/20/2014http://srdr.ahrq.gov51  It is preferred that Arms and Outcome information be submitted as part of the project’s extracted study data.  Structure of Spreadsheets - Individual row for each study - Individual column headers for each Arm (for Arms Spreadsheet) or Outcome (Outcomes Spreadsheet) that each study is assigned to - 1 or 0s delineate the Arms or Outcomes included in each study Example Arm and Outcome Spreadsheets Outcome SpreadsheetArm Spreadsheet

52. 6/20/2014http://srdr.ahrq.gov52  The method:  Data is entered retrospectively into SRDR using source file(s) from the report to directly import extracted study data into SRDR  Data saved in Microsoft Excel source file(s) for each study in the report is directly imported into SRDR by the SRDR team  Materials:  Microsoft Excel source files containing extracted study data with specific formatting  Arms and outcome spreadsheets to be included with each source file containing arms and outcomes data  Characteristics:  Extraction form templates with any number of datafields  Large report size, including any number of eligible studies Publish the project to SRDR’s published projects page Submit source file(s) to SRDR team for import into the project Prepare source file(s) for import Input general project information and key questions Initiate a project in SRDR 1 5 4 3 2 Method 3 - summary

53. 6/20/2014http://srdr.ahrq.gov53 Overall Method Summary Method 1 SRDR interface Method 2 SRDR extraction form templates Method 3 Direct import of source file(s) Import type Manual, EPC teamExport, EPC teamDirect, SRDR team File type Any Excel Datafields Limited Any Report size SmallMedium Large # steps 68 5

54. 6/20/2014http://srdr.ahrq.gov54 Presentation outline The definition of prospective and retrospective data entry into SRDR The benefits and challenges related to entering data, retrospectively, into SRDR Method 1: using the SRDR interface Method 2: using SRDR exported extraction form template(s) Method 3: using source file(s) directly imported into SRDR Future directions

55. 6/20/2014http://srdr.ahrq.gov55  The SRDR team is currently developing a feature that will allow EPC teams to retrospectively enter data from their EPC reports by importing their source files directly into SRDR, without having to first submit their source files to the SRDR team.  Use of this this feature would effectively replace the involvement of the SRDR team in step 4 of Method 3 for entering data retrospectively into SRDR Future Directions

56. 6/20/2014http://srdr.ahrq.gov56 Future Method - steps Publish the project to SRDR’s published projects page Submit source file(s) to a new SRDR tool to import data into the appropriate data fields on SRDR extraction forms in the project Prepare source file(s) from the report to be directly imported into SRDR Input general project information and key question(s) from the report Initiate a project in SRDR 1 5 4 3 2

57. 6/20/2014http://srdr.ahrq.gov57 Brief demonstration of simple exporting of an SRDR extraction form template within the advanced data export feature of SRDR