1. National Institutes of Health U.S. Department of Health and Human Services Planning for a Team Science Evaluation ∞ NIEHS: Children’s Health Exposure Analysis Resource (CHEAR) Program Kristi Pettibone, PhD & Jenny Collins | NIEHS

2. National Institutes of Health U.S. Department of Health and Human Services NIEHS Mission: The mission of the National Institute of Environmental Health Sciences is to discover how the environment affects people in order to promote healthier lives. The Division of Extramural Research and Training (DERT) funds a broad portfolio of research across the country and abroad National Institute of Environmental Health Sciences

3. National Institutes of Health U.S. Department of Health and Human Services New Resource for Exposome Research Children’s Health Exposure Analysis Resource (CHEAR)

4. National Institutes of Health U.S. Department of Health and Human Services New Resource for Exposome Research Children’s Health Exposure Analysis Resource (CHEAR)

5. National Institutes of Health U.S. Department of Health and Human Services Purpose Provide access to labs, tools and data analyses So that researchers can expand their research to include more analysis of environmental exposures in children’s health research

6. National Institutes of Health U.S. Department of Health and Human Services Intended Users Children’s health researchers with limited experience analyzing environmental exposures, or those with some data, but seeking more extensive analyses.

7. National Institutes of Health U.S. Department of Health and Human Services CHEAR Infrastructure Work together to develop tools, conduct analyses, and promote resources to other researchers. Steering Committee Coordinating Center National Exposure Assessment Laboratory Network Hub 1 Data Repository, Analysis and Science Center Hub 2 Hub 3 Hub 4 Hub 5 Hub 6

8. National Institutes of Health U.S. Department of Health and Human Services CHEAR Outcomes Expand the number of studies that include environmental exposure analysis in their studies, Implement the exposome concept in children’s health studies, Develop data and metadata standards for the environmental health sciences community, Create a public resource of children’s exposures across the country.

9. National Institutes of Health U.S. Department of Health and Human Services Goals of Today’s Session Describe team science approaches used to facilitate the program goals Describe team science focused evaluation goals and questions planned for the program Obtain feedback on the appropriateness of the evaluation questions and metrics, and ideas for additional metrics

10. National Institutes of Health U.S. Department of Health and Human Services Team Science Approaches to Facilitate Program Goals Data Coordinating Center Time for Team Building Outcome and Process Evaluation Steering Committee/Governance Rules Clarifying Program Goals, Objectives, Impacts Timeline Communication

11. National Institutes of Health U.S. Department of Health and Human Services Team Science Approaches: Leadership Steering Committee Coordinating Center National Exposure Assessment Laboratory Network Hub 1 Data Repository, Analysis and Science Center Hub 2 Hub 3 Hub 4 Hub 5 Hub 6

12. National Institutes of Health U.S. Department of Health and Human Services Team Science Approaches: Time for Team Building Goal for the first year is getting the team in place and get the “doors open” Also built in time for team building, bonding, etc. Before we do the trust falls, check the records to see if there is a lawyer in the group.

13. National Institutes of Health U.S. Department of Health and Human Services Team Science Approaches: Steering Committee/Governance Rules Steering Committee convened before first grantee meeting. Addressed voting, roles and responsibilities, meetings and logistics.

14. National Institutes of Health U.S. Department of Health and Human Services Team Science Approaches: Outcome and Process Evaluations Evaluation was built into the process from the beginning. Grantees had opportunity to provide input into the evaluation design.

15. National Institutes of Health U.S. Department of Health and Human Services Team Science Approaches: Clarifying Program Goals & Impacts

16. National Institutes of Health U.S. Department of Health and Human Services Team Science Approaches: Timeline

17. National Institutes of Health U.S. Department of Health and Human Services Team Science Approaches: Communication

18. National Institutes of Health U.S. Department of Health and Human Services Improved understanding of environmental exposures on child health and development Increased opportunities for data pooling and comparative analyses Expanded capabilities or capacities for measuring chemical and non-chemical stressors that affect children’s well-being and development Number of co-funded publications Increased number of collaborative research projects that use CHEAR data or tools Establish “normal” measurements across age, gender, race/ethnicity to allow interpretation of results in unique cohorts. Increased number of children’s health studies that include environmental exposure analysis Increased number of data and metadata standards for the environmental science community Increased interoperability of exposure data with data generated within and outside the network Increased statistical power for analyses Shortened time frame for findings in children’s environmental research Change in number of applications with children’s EH RCDC codes from all ICs Increased inclusion of comprehensive exposure analysis in children’s health research Increased communication among researchers Increased number of children’s health studies that incorporate exposome concept Number of satisfied customers Number of referrals Number of repeat customers Number of R01s funded based on CHEAR data Number of customers/PIs without NIEHS funding who are using CHEAR Number of citations of CHEAR data CHEAR Impacts and Metrics Impacts Metrics

19. National Institutes of Health U.S. Department of Health and Human Services Possible Consortium Evaluation Questions Accomplishments of the consortium efforts (both methods and findings) Products (e.g., joint publications, presentations, collaborations, datasets, new grants) Satisfaction with this project? Experience with this consortium model versus other funding models? Benefits with funding this research through a consortium? Challenges associated with funding this research through a consortium?

20. National Institutes of Health U.S. Department of Health and Human Services Possible Consortium Evaluation Questions Possible re-use of TREC consortium evaluation questions –Expectations about participation in research activities –Experience with consortium activities prior to CHEAR –Engagement in activities that promote or facilitate trans-disciplinary work –Views on collaboration –Experience with disagreement –Perspective on team success

21. National Institutes of Health U.S. Department of Health and Human Services Questions and Comments What other metrics have you used to assess team science initiatives? How do you balance federal need for data on outcomes, with role of coordinating center, and response burden? How have you used progress reports or grantee meetings to obtain information on the team science process? How do you use anecdotal information or observations from grantee meetings about how well things are working? How do you get grantees to engage in the team science project? Put aside their own agenda and bring their contributions to the greater agenda? How do you evaluate team science as an NIH institute when you fund grantees to do scientific research, not team science research?

22. National Institutes of Health U.S. Department of Health and Human Services Acknowledgements CHEAR Grantees CHEAR Co-Directors –David Balshaw – Exposure, Response, and Technology Branch Chief –Claudia Thompson – Population Health Branch Chief Jenny Collins, CHEAR Program Analyst Contact Information: Kristi Pettibone pettibonekg@niehs.nih.gov 919-541-7752