Please Sit with Someone You Don’t Know Before we begin, write down what success means for one of your projects or initiatives Determining Evidence of Success 1

Determining Evidence of Success: How Do You Know and Why Are You Sure? Deborah Boisvert, PI for SynergyRPT Elaine Johnson and David McNeel, CoPIs

Initial Dialogue Person A: Share a bit about the project or initiative you chose with Person B. 3

Initial Dialogue Person B: Tell Person A what evidence you would need in order to know if their project or initiative were successful. 4

Initial Dialogue Person B: Share a bit about the project or initiative you chose with Person A. 5

Initial Dialogue Person A: Tell Person B what evidence you would need in order to know if their project or initiative were successful. 6

SynergyRPT What have we learned through SynergyRPT about measuring outcomes? 7

Outputs vs... It’s Easy to Measure Outputs 124 student participants 53 workshop attendees 16 curriculum modules 8

… Outcomes Outcomes Are a Different Matter: An outcome is the “So what?” of the outputs (individually and cumulatively) 124 Student Participants - Did they learn; did they complete the course of study; did they continue in the same field of study? 53 Workshop Attendees - Were the activities useful and relevant; did they meet a need? 9

SynergyRPT Community of Practice How did the participating Centers address the challenges of outcome data collection and analysis? 10

Assessing Outcome Data Collection and Analysis Outcomes to be measured should be: Sharply defined Distinct from outputs Measurable 11

Assessing Outcome Data Collection and Analysis Data to be collected should be: Behavioral Narrowly focused on the outcome Longitudinal to the extent possible 12

Assessing Outcome Data Collection and Analysis The measurement process should employ: Multiple methods Measurements taken from multiple perspectives 13

Assessing Outcome Data Collection and Analysis Analysis and conclusions should: Use contextual data Identify areas for improvement Disprove other possible conclusions 14

Assessing Outcome Data Collection and Analysis A Case Study from Broadening Advanced Technological Education Connections (BATEC): Tech Apprentice Internship Model 15

Assessing Outcome Data Collection and Analysis A Case Study from Bio-Link’s Next Generation National Advanced Technological Education Center for Biotechnology and Life Sciences: Bridge to Biotech 16

Let’s discuss and reflect Can you have “success” without well-defined outcome(s)? When can/should you change an outcome for your project or initiative? What would it take for you to understand your success (or not) and understand WHY you were successful (or not)? 17

Initial dialogue revisited Review your initial dialogue with your partner in light of this presentation : Are you measuring outputs or outcomes? Is your outcome measurable? What are your methods for selecting, capturing and analyzing outcome data? Are you measuring evidence or value of success 18

Outcome Data Collection and Analysis Questions and Concluding Thoughts ??? 19