1. caDSR Enablement of PRESAGE Fox Chase Cancer Center

2. Purpose To describe Fox Chase Cancer Center’s experiences in enabling its existing PRESAGE toolset to access Common Data Elements (CDEs) in the caDSR to generate web-based data entry forms.

3. PS-SIG Needs Facilitate both information collection and data sharing the PS-SIG recommended the creation of a standard for describing data collection instruments. The standard should include: 1.Grouping and order of instruments and questions; 2.All skip-pattern and branching logic; 3.Quality assurance checks for data type, range, permissible values and logical consistency; 4.Separation of questions from their flow and presentation. Ideally the standard would be sufficiently complete such that two systems could produce a data collection instrument that was identical in content and data quality assurance.

4. Benefits of Enablement Data Sharing Data exchange frequently requires mapping from local CDEs to a common, agreed upon list of CDEs. –This process can be very time consuming. –Leads to loss of information as local CDEs and valid values are “squeezed” into exchangeable definitions. –Small differences in the way questions and responses are worded, or presented, may lead to significant (potentially unrecognized) differences in interpretation.

5. Benefits of Enablement Improve Research Efficiencies and Data Quality Selecting valid questions and values can be time consuming. Simplified access to caDSR CDEs could reduce the time it takes to select and build new survey instruments. The use of curated CDEs that contain complete data validation constraints and skip logic should improve data quality. The ability to create and store all metadata needed for a fully functional data entry tool could substantially improve the efficiency and consistency of population research. This should be combined with tools (like PRESAGE) that can use this metadata to easily create fully functional data collection products. Direct entry by study participants though on-line tools should reduce transcription and keypunch errors associated with entry from paper forms. It is our anecdotal experience that web-based data collection reduces missing data and data quality issues. Collaboration

6. Prototype Development

7. Use Cases 1.Create a web-based questionnaire for data capture from caDSR metadata. 2.Generate a printable completed questionnaire.

8. Prototype Requirements Import caDSR Metadata into PRESAGE Add Pages to Form Modules Add and Order Questions on Pages Add Skip Patterns to Pages Link Questions (Multi-part and Likert-style) Add Question Presentation Metadata Add Valid Value and Range Checking Metadata Export Survey XML data Make any necessary enhancements to PRESAGE to support the NHIS 2005 questionnaire Generate web-based survey Print completed survey

9. Export of Case Report Form Metadata “2005 NHIS Questionnaire – Sample Adult” case report form (CRF) from the NCI Population Sciences & Cancer Control (PS&CC) context was used for this project. Prior to this pilot project this form and all required CDEs were curated by Mary Cooper (PS&CC context curator).

10. Export of Case Report Form Metadata Considered two ways to extract metadata from the caDSR: Excel export and the caCORE API. Both approaches have advantages and disadvantages. AdvantagesDisadvantages Excel Export easy to obtain could be worked with “off line” missing some important form metadata –Skip pattern –CDE validation metadata caCORE API provides complete access to all the CDE and FormBuilder metadata (we think) documentation was incomplete for much of the API (particularly skip patterns) accessing NCICB servers was also slow

11. PRESAGE Data Entry Interface Builder

12. Modules/Instruments

13. Pages

14. Questions

15. Implementing the NHIS Question Presentation Types

16. Implementing the NHIS Skip Patterns - FormBuilder Supports only simple skip patterns –Valid value from an enumerated list –Skip target can be a form, module or question on either the current form or a different form Skip patterns can also include free text instructions Severely limits the utility of FormBuilder

17. Implementing the NHIS Skip Patterns - PRESAGE Simple (similar to FormBuilder) Complex More Complex ConditionDescriptionExample Usage CompareCompare the response provided to a value  The number of times per week provided is greater than 3  If age provided is less than 16 BlankReturns true if no response was given to the question If the participant didn’t provide a response to the question. Sum of QuestionsSums the numeric responses to a list of questions and compares the answer to a value If the responses should equal 100 percent. Randomization ArmSkip questions based on randomization armDid you find the health education presentation useful? Number of YearsCalculate how many years have passed since a given date Used in eligibility determination. If it has been more than 6 years since the cancer diagnosis.

18. Data Validation Data type: Ensure the value provided is of the correct data type (e.g., number or date). Format: Ensure the format of the value is what was expected (e.g., date formatted as MM/DD/YY). Ranges: Ensure that the value lies within the range of acceptable values (e.g., age is between 0 and 120). Required: Checks that a value was provided. Consistency: Checks for inconsistencies between data values (e.g., prostate cancer in a female).

19. Implementing the NHIS Data Validation - FormBuilder FormBuilder provides extremely limited data validation capabilities –Only supports an indicator as to whether the answer is mandatory. CDE validation metadata: –Lists of valid values –Data type –Minimum and Maximum –Format

20. Implementing the NHIS Data Validation - PRESAGE Simple (Field-Level Validation) Complex (Multi-field, Consistency Checks) Created specific custom validations –For the question “How old were you when you first started to smoke regularly?” error when the response is greater than age except if the response is 96 (never smoked regularly) –For the question “How long has it been since you quit smoking cigarettes?” error when the value provided is greater than the age minus 15, or when the age started plus the value provided is greater than the person’s age

21. PRESAGE Survey XML Represents PRESAGE Survey and Question Metadata Used by Data Entry Interface Code Generators

22. PRESAGE Survey

23. Hardcopy Printed Survey PRESAGE produces a data dictionary, or codebook, for each survey. We added the ability to combine metadata and response data to produce a printable copy of the answers provided. This report can be used for: –quality assurance, –paper audit trail, –review by healthcare providers or study participants.

24. Project Findings Desire: caDSR as a standard for describing data collection instruments. Standard should: –Describe grouping and order of instruments, forms, and questions. –Describe all skip-pattern and branching logic. –Describe data quality assurance measures including variable type, range, permissible values, and logical consistency checks, etc. –Be developed in a fully modular fashion that separates the questions from their flow and presentation on a form or screen.

25. Project Findings “Show Stoppers” Provide support for multi-part questions. Provide a method for expressing more complex data validations. Provide a method for expressing more complex skip patterns. Include question presentation metadata. Include all relevant metadata in form export files. Improve documentation for APIs related to CRF metadata.

26. Project Findings “High Priority” Capture of copyright or licensing information. Provide support for grouping of questions into study instruments that can be combined into questionnaires. Provide a unique, immutable identifier for each CDE and FormBuilder component. Provide support for dynamic content within questions. Group questions so they can be displayed together. Provide support for storing data validation error messages. Provide support for validation with multiple non- contiguous ranges.

27. Project Findings “Medium Priority” Capture delivery method (CATI, CAPI, mailed, email). Provide a method for versioning questions separate from CDE versioning. Provide support for additional formatting markup (bold, italics, underline). Provide support for multiple valid value meaning texts. Provide ability to specify a variable name to be used in analysis packages.

28. Project Findings “Further Research and Consideration Needed” Consider support for “actions” associated with skip patterns. Provide a method for identifying potential collaborators. Provide support for hard vs. soft validation error messages. Capture question numbers.

29. Conclusions Demonstrated it is possible to use CDE and FormBuilder metadata to generate web-based questionnaires using PRESAGE. caDSR metadata alone was not sufficient to produce this questionnaire. Resolution of the issues raised would constitute an important first step in increasing the use of standards in this community. We do not discuss the multitude of cultural and social challenges that face the adoption of the CDEs and Forms. These non-technical hurdles may prove more difficult to resolve than the technical issues.