1. Using Collaborative/Participatory Evaluation Methods to Give Voice to Stakeholders in Clinical and Translational Research
AEA 2018
Chair: John F. Stevenson, University of Rhode Island
Discussant: William Trochim, Cornell
"Translational" = Research steps to expedite the movement from laboratory research to impactful health practices
John Stevenson, Chair
William Trochim, Distinguished Discussant

2. Acknowledgement
Research reported in this session was supported via Institutional Development Awards (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Thanks to NIGMS for support helping to make this research possible!

3. IDeA-CTR
Purpose:
To enhance biomedical research activities in states that have had historically low NIH grant funding success rates
Provides support for:
Developing infrastructure and human resources required to conduct clinical and translational research in IDeA-eligible states
Enhancing the ability of IDeA institutions and investigators to develop competitive clinical and translational research programs
Fostering and sustaining collaboration and coordination of clinical and translational activities within and across IDeA institutions and organizations
Addressing health conditions that affect the medically underserved and/or that are prevalent among populations in IDeA states
For states that have had historically low NIH grant funding success rates.
These CTR Centers
provide the infrastructure, resources , professional development that will
enhance the competitiveness of the investigators
to obtain additional funding for clinical and translational research. 
and enhance collaboration and coordination

4. Required Key Component Activities (KCAs)
Biostatistics Clinical, Epidemiology, and Research Design Core
Professional Development Core
Pilot Projects Program
Tracking and Evaluation
Community Engagement and Outreach Core*
Terminology "KCA" helpful for today's presentations
These activities make good sense given the mission
This list has evolved. Most sites have added optional KCAs.

5. IDeA Program Infrastructure for Clinical and Translational Research (IDeA-CTR)
10 centers are currently funded (maybe already out of date!?)
States represented:
Mississippi
Nebraska
Rhode Island
South Carolina

6. Themes for This Session
1. Collaborative/participatory methods to engage stakeholders in evaluation
Recommended by the Evaluation Committee of a CTSA Consortium
Supported by the empirical meta- evaluation literature
Captured in 8 evidence-based principles
Preaching to the choir re: partic/collab for utilization
Shula et al (2016 AJE): these principles helpfully organize thinking and acting for a collab approach
Shulha, L.M., Whitmore, E., Cousins, J.B., Gilbert, N., & al Hudib, H. (2016). Introducing evidence-based principles to guide collaborative approaches to evaluation: Results of an empirical process. American Journal of Evaluation, 37(2),
Clarify motives for collaboration
Foster meaningful relationships
Develop a shared understanding of the program
Promote appropriate participatory processes
Monitor and respond to resource availability
Monitor evaluation progress and quality
Promote evaluative thinking
Follow through to realize use

7. Themes for This Session
2. Innovative methods for conveying stakeholder perspectives to key policy-makers
Complex context: multiple institutions and hierarchic levels of power within them
Varying organizational and professional cultures
“Stakeholders” can include novice researchers, administrators, as well as engaged “communities”
Internal and external evaluation perspectives
Speaking truth up the hierarchy and across organizations with differing cultures

8. Today’s Presentations
Using Evaluation Retreats for Iterative and Collaborative Review
L. Boney, S. Downey, & M.L. Lindsey
Evaluation to Action: Quarterly Reports and Continuous Quality Improvement
P. Estabrooks, M. Cramer, & J.J. Rohde
Increasing the Quality and Quantity of CTR in RI: Comparing Views of Program Leadership and the Local Research Community
J.Fede, S.J. Kogut, J.F. Stevenson, A.R. Hayward, & C. Willey
Maintenance of Translational Evaluations with a Toolkit of Theories
R. Paranal & J. Harvey
Best to move straight into the first paper -- Sue Downey for Mississippi

9. Audience participation:
Hold discussion questions and comments until all presenters and the Discussant have finished
Ask clarification questions at any time

10. Today’s Presentations
Using Evaluation Retreats for Iterative and Collaborative Review
L. Boney, S. Downey, & M.L. Lindsey
Evaluation to Action: Quarterly Reports and Continuous Quality Improvement
P. Estabrooks, M. Cramer, & J.J. Rohde
Increasing the Quality and Quantity of CTR in RI: Comparing Views of Program Leadership and the Local Research Community
J.Fede, S.J. Kogut, J.F. Stevenson, A.R. Hayward, & C. Willey
Maintenance of Translational Evaluations with a Toolkit of Theories
R. Paranal & J. Harvey