1. NCI Behavioral Research Measures Project Tom Milke Senior Systems Analyst Westat Seattle Oct 25, 2007

2. Foundations for Enablement of Cyberinfrastructure* Community Agreement on Vocabulary, Structure of Measures Structured Data Systems, Interoperability, Common Messaging Standards Integrative Applications *Courtesy of Christopher Greer, NSF

3. State of Behavioral Science* Community Agreement on Vocabulary, Structure of Measures Structured Data Systems, Interoperability, Common Messaging Standards Integrative Applications *Courtesy of Christopher Greer, NSF Behavioral Research

4. Current Landscape of Measures Public Health Surveillance National Surveillance Data (e.g., NHIS, BRFSS, NHANEs) Routine behavioral surveys (e.g., HINTS) Field based data University Laboratories Individually published papers Locally maintained data sets Medical Research Settings Patient Charts Clinical trials data Local Interventions

5. Measurement Concepts in Behavioral Medicine Nicotine Dependence Construct Measure *i.e., measures can be used interchangably to assess the construct. Usage would vary depending on population, modality, and purpose. Alternative Measures with Psychometric Equivalencies*

6. BRP Proposal: Create a Database of GRID Compatible Measures Construct Measure http://dccps.cancer.gov/brp/constructs/index.html Health Behavior Measures

7. Purpose of the Measures Database:  Create an online resource of commonly used behavioral measures in cancer control  Facilitate access by applicants, grantees, and other scientists.  Help the Behavioral Research Program (BRP) realize its mission of encouraging rigorous, replicable research applied to behavioral issues in cancer control.

8. 8 Phase I Environmental Scan  A static resource was already available through the BRP web site.  Westat conducted a pilot project to do an environmental scan of comparable resources currently available.

9. 9 Results of Environmental Scan  Identified illustrative web sites that routinely make measures available to their respective communities  Profiled sites on set of criteria  Identified current practices  Made recommendations for how to structure the BRP measures site  Developed comprehensive list of attributes about measures to offer to researchers for identifying measures of interest  Wireframes to show structure and elements for UI

10. 10 Moving Forward  Next step would be to develop a small, testable prototype to illustrate feasibility.  Ideally, the prototype should follow the principles associated with caBIG tool development.  The blueprint should be articulated using Unified Modeling Language (UML);  The data should be discoverable on “The Grid”.  The semantic structure of the elements should correspond to NCI’s Enterprise Wide Vocabulary as instantiated in the caDSR.

11. 11 Moving Forward  The measures study team recognized that many population science variables do not currently exist in the caDSR.  One of the outcomes of the project should be to gain a better understanding of the process needed to register Population Science data elements within the global caDSR or, alternatively, in a DCCPS-managed subset of the caDSR data space.

12. 12 Moving Forward – Questions/considerations  What would it take to incentivize usage and to construct an online community resource?  Should the process be top-down, a resource-intensive activity that would make BRP a Clearing House for vetted measures; or should it be “bottom-up,” using social technologies to facilitate community data sharing?  Preference for creating a self-perpetuating resource that is driven and updated by the extramural community.  Bob Croyle pointed to the success of online knowledge sharing tools such as Wikipedia and PubCan as models.

13. Use Case Grantee required to use CDE’s Browses measures based on construct, population, mode, psychometric qualities Selects measure for use in grant submission, research Provisions for adding new measures Community vetting (e.g., usage statistics) Health Behavior Measures

14. Use Case Diagram

15. 15 Domain Model

16. 16 The Measures Class and Its Attributes

17. 17 Ancient Chinese Classification of Animals  The Analytical Language of John Wilkins," by Borges describes the Celestial Emporium of Benevolent Knowledge, in which it is written that animals are divided into:John WilkinsBorges  those that belong to the Emperor,  embalmed ones,  those that are trained,  suckling pigs,  mermaids,  fabulous ones,  stray dogs,  those included in the present classification,  those that tremble as if they were mad,  innumerable ones,  those drawn with a very fine camelhair brush,  others,  those that have just broken a flower vase,  those that from a long way off look like flies.

18. 18 We want to store and search for constructs: Challenge at the vocabulary level What’s a Construct? (Precoordinated terms) Adolescent YouthTeenCategorized Age SmokingDependence

19. 19 Functions/Features Under Consideration  Search Capability (simple, advanced, index)  Resources  Measure comparison tool  Helps (how to evaluate measures, ways to frame searches)  Related links  FAQs  Community/Sharing  Measure submission process  Reviews (read and write)  Notifications  “Most viewed…” measures/constructs  Recommendations  Personalized workspace  Saved search results, Personal reviews,  Personalized notifications

20. 20 Search Constructs & Measures Share/Community Quick Links New! Help Contact Us Search Constructs & Measures Evaluate Measures Share/ Community Search Constructs & Measures Resources/ Help and Links My Research Space Researcher Resources - Your search yielded the following results: All measures Show only downloadable measures ∙ Number of Recommendations WISDM -68 Previous 1 2 3….Next xxxxxxxxxxxxxxxxx Search Compare Measures NDSSxxxxxxxxxxxx xxxxxx xxxxx xxxxxxxxx CDS - 5 Measure Author Target Population Mode of Admin. Number of Items iTunes Model

21. 21 Share/Community Search Constructs & Measures Resources/ Help and Links My Research Space Researcher Resources Quick Links Help Contact Us New! Evaluate Measures Search Constructs & Measures - - Compare Measures: Select the measures you would like to compare. Number of Recommendations WISD M-68 Previous 1 2 3….Next xx x xxxxxx xx x xxxxx Target Population Mode of Admin. Number of Items x x Compare NDS S Author CDS - 5 Search Measure xx x xxxxxx xx x xxxxx Compare Measures

22. 22 Select Attributes to Compare Share/Community Quick Links Help Contact Us New! Evaluate Measures Search Constructs & Measures Resources/ Help and Links My Research Space Researcher Resources Share/ Community Search Constructs & Measures - - Compare Measures: Add attributes of the measure that you would like to compare: Author # of Items Mode of Admin. Target Population Source References >> << All AttributesAttributes to Compare Continue Cost Data sharing capability Author Contact Info. # of Recommendations

23. 23 Compare Measures Share/Community Quick Links Help Contact Us New! Evaluate Measures Search Constructs & Measures Resources/ Help and Links My Research Space Researcher Resources Share/ Community Search Constructs & Measures - - Comparison of Measures: NDSSWISDM-64 Cost Data Sharing Capability Author Contact Information Number of Recommendations CDS-5 xxx xxxxxx xxx Attributes Search

24. 24 TAB Model NCI Header NCI Left Nav. All Researcher Resources My Research Space Help and Links Researcher Community Search Constructs and Measures Dropdown... Search Constructs and Measures: simple search,advanced search, Index, guide, compare Measures, Researcher Community: "sharing,"most viewed,"people who liked X liked Y...",reviews,site updates, RSS,tag clouds Resources,Helps and Links: links to other sites,tips on how to evaluate measures,help files,FAQ My Research Space: profile,my recent searches,my reviews, " people like ME liked Y...“, site updates Go Searchfor [free text] Dropdown list is"all measures and constructs," measures,"and "constructs"--default is to search all measures and constructs. RSS feeds, notifications of new content, tag clouds, guide to search

25. Discussion

26. Discussion Points What are the sociotechnical solutions to the “common data element” problem? E.g., –Incentivized data sharing through initiatives? –Specialized caDSR? –Vetting bodies for standards in population science? Where are targets of opportunity? –Federal surveillance? –Clinical protocols? –Behavioral measures in electronic health records?

27. Discussion Points What tools are needed to enable transformations in: discovery, visualization, decision support, fusion, and policy planning?

28. Discussion Points Can Pop Sci SIG create a “blueprint” to guide growth strategically? –Overall vision? –Needed standards investments? –Needed tool investments? –Ethical, social, and legal concerns? –GRID level use case scenarios

29. 29 FINIS  Rick Moser moserr@mail.nih.govmoserr@mail.nih.gov  Brad Hesse hesseb@mail.nih.govhesseb@mail.nih.gov  Jennifer Crafts jennifercrafts@westat.comjennifercrafts@westat.com  Tom Milke tommilke@westat.com