1. Cancer Disparities Research Partnership Program Process & Outcome Evaluation Amanda Greene, PhD, MPH, RN Paul Young, MBA, MPH Natalie Stultz, MS NOVA Research Company

2. 2 Presentation Outline Evaluation Purpose Evaluation Timeline Conceptual Framework Evaluation Question Examples Data Collection and Analysis Evaluation Products

3. 3 Purpose of Evaluation Measure relevance, effectiveness, and impact of CDRP Program in a consistent fashion Evaluation findings to identify Program attributes to be applied to other settings Meaningful annual reports for stakeholders within and outside the NCI

4. 4 Type of Program Evaluation 3-year, 9-month evaluation Process components Is program being implemented as planned Is targeted audience being reached Outcome components Were short-term and intermediate objectives achieved

5. Evaluation Timeline

6. 6 CDRP Goal Build and stabilize research capacity in radiation oncology in community-based institutions that care for a disproportionate number of medically underserved, low-income, and racial and ethnic minority populations.

7. 7 CDRP Program Conceptual Framework

8. 8 Evaluation Design: Focus of Key Questions Improvement and conduct of radiation oncology clinical research in community- based health care institutions serving large populations experiencing health disparities Clinical trial participation by target populations Influence of partnerships between awardee institutions and academic research centers Influence of TELESYNERGY ® Influence of Patient Navigation

9. 9 Clinical Trials Examples of evaluation questions Did the CDRP Program result in increased clinical and translational research with populations experiencing cancer health disparities? Have community-based clinical researchers improved their knowledge, attitudes, & practice (KAP) of radiation oncology research? Outcomes Increased participation in clinical trials by target populations Improved KAP by local researchers Increased number clinical trials available at local site Publications and presentations about findings by local researchers

10. 10 Partnerships Examples of evaluation questions What is the nature of collaboration between grantees & partners? What is the influence of collaboration on clinical research and treatment outcomes at the grantee sites? Outcomes Increased frequency and quality of communication between partners related to clinical research and patient consultation Increased collaboration (clinical trials, research contacts, patient consultations) among academic and community- based clinical researchers.

11. 11 TELESYNERGY® Examples of evaluation questions What is the influence of TELESYNERGY ® on building partnerships, facilitating clinical research, and improving treatment outcomes? Outcomes TELESYNERGY ® facilitates patient access to care, clinical decision making and diagnosis. TELESYNERGY ® increases research collaboration between community sites and their academic partners report.

12. 12 Patient Navigator Program Examples of evaluation questions Has Patient Navigation facilitated participation of minorities in clinical trials, improved access to cancer care services, and impacted patient outcomes? Outcomes Increased participation in clinical trials by target populations Patients’ barriers to cancer care (e.g., financial, cultural, transportation) reduced. Patients adhere to treatment protocols.

13. 13 Target Populations for Evaluation Data Collection Cancer patients from targeted populations experiencing cancer health disparities Principal Investigators, participating radiation oncologists, and awardee institution staff Partners from academic research centers Patient Navigators

14. 14 Data Sources Archival Data Program-related written documents PI meeting minutes, organizational records and charts Historical data on local cancer incidence, prevalence, mortality Baseline data on clinical trials and research activities at participating sites New Data In-depth structured interviews with PIs and partners Patient focus groups Group interviews with Patient Navigators and Navigator support staff Surveys on issues related to recruitment to clinical trials

15. 15 Comparison Group 6 similar community-based radiation oncology facilities Comparison sites will be chosen based on same criteria as stated in RFA Conduct brief survey about independent and collaborative clinical research capabilities and current participation in radiation oncology clinical trials

16. 16 Data Analysis Each program component will be analyzed Clinical trials Partnerships between awardee and academic partner TELESYNERGY® Patient Navigation Sources of information will be combined to analyze overall Program and linkages between/among components.

17. 17 Data Analysis (continued) Quantitative Descriptive statistics Complex analyses and causal modeling Resource cost-allocation analysis Qualitative—Thematic Analysis Intended and unintended successes Failures Critical incidents Lessons learned Recommendations for future programs

18. 18 Products of Evaluation: Reports Annual process/progress reports Program implementation, accomplishments, and progress to date Needed midcourse changes to meet objectives. Final outcome report Evidence about Program’s attainment of goals and expected outcomes. Summarize plausible mechanisms of change Delineate temporal sequences between activities and effects