Continuous Improvement Collecting, Analyzing, and Sharing Data

At the end of this presentation, you will know: 1)All the wonderful ways that you can use data for important purposes 2)What “formative evaluation” is, and how it relates to continuous improvement 3)How to organize yourselves for formative evaluation 4)How to collect data and use it to fuel continuous improvement 5)Different approaches to analyzing, reporting on, reflecting on, and sharing your data 6)How to work with C-PAD to accomplish data- related tasks

The Joys of Data!

Data are not just for grant requirements anymore! Continuous Improvement Marketing your Program Positioning your Program for Funding

Participation Satisfaction Tailoring Effectiveness By using data for continuous improvement, you will increase:

Using Formative Evaluation for Continuous Improvement

Evaluation = using data for assessment and learning Formative = for the purposes of creating, developing, and improving Formative (program) evaluation = Using data to inform how you can continually pursue program excellence What is formative evaluation?

Bullet Points Formative Evaluation as a Tool for Continuous Improvement Formative evaluation asks: So that your team can further investigate: In order to achieve: Is the program reaching its participation targets? How can we attract more people to the program? Increased participation Are participants satisfied with the offerings? How can we increase satisfaction levels? Increased satisfaction Does the design of services meet the needs of its participants? How can we better align service offerings with participant needs? Increased tailoring Do participants succeed in desired outcomes? How can we increase the rate of positive outcomes? Increased effectiveness

Forget: “evaluation” Remember: “leveraging data for program excellence!” We are busy delivering a program – how can we be evaluators too?

Julie Seeley, Spoon River College Story from the Field: Leveraging data for program excellence

The Formative Evaluation Toolkit: A Guided Tour

Many pages (but not to worry!) Step-by-step guidance through the phases of evaluation Tools and templates for conducting your evaluation Templates for working meetings to accomplish evaluation tasks Checklists at the end of each chapter to keep you on track Blank pages for notes, questions, and insights Support for: –Completing deliverables due to Champion Colleges –Fulfilling grantee reporting requirements What you will find in the FET:

Data collection (chapters 3 & 4) Organizing for evaluation (chapter 2) Reporting on findings (chapter 5) Reflecting on findings (chapter 5) Step-by-step guidance for:

Data Collection (Chapters 3 and 4)

Process data Outcome data Stakeholder feedback Data Types

Counts of program participants –Students completing workforce program courses –Students participating in support services Information about program components, describing: –Workforce programs and courses –Math, English, and computer courses Process Data (Chapter 3)

Number of students who have: –Received credits for prior learning –Received a degree –Received a certificate –Received a non-credit certificate –Become employed Outcome Data (Chapter 3)

“Extractable” v. “Real-Time” Data Make a data collection plan –Identify the data you need to collect –Figure out how you’ll collect it Work with an IR Partner Plan early for real-time data collection Stay organized with data storage Important Considerations for Collecting Process & Outcome Data

Guidance for a working meeting to develop process and outcome data collection plans Tools: –Tool 2: Data Choices Table (p. 42) –Tool 3: Process Data Collection Plan Worksheet (p. 45) –Tool 4: Outcome Data Collection Plan Worksheet (p. 54) –Tool 5: Options for Collecting Employment Data (p. 57) –Data Storage Excel Workbook (URL on p. 17) FET Tools for Process and Outcome Data

Hearing from students, internal partners, and community partners Looking for feedback that will help you to: –Attract more students to the program –Tailor services better –Raise satisfaction levels –Support students better for completion and employment Stakeholder Feedback (Chapter 4)

Decide what you want to learn Make a data collection plan –Decide which stakeholders you want to hear from –Identify the data collection methods you want to use –Identify people responsible for collecting feedback Make data collection instruments Don’t collect too much data! Important Considerations for Collecting Stakeholder Feedback

Guidance for two working meetings to develop: –A stakeholder feedback data collection plan –Data collection instruments Tools: –Tool 6: Stakeholder Feedback Evaluation Questions Worksheet (p. 61) –Tool 7: Stakeholder Feedback Data Collection Plan Worksheet (p. 63) –Tools 8 & 9: Survey, Interview, and Focus Group Protocol Templates (pp. 66, 82) –Tools 10-12: Guidance for Conducting Interviews and Focus Groups (pp. 88, 90, 91) FET Tools for Stakeholder Feedback

Organizing for Evaluation (Chapter 2)

Evaluation Team Manager IR Partner Process and Outcome (P&O) Data Lead Stakeholder Feedback Lead Assembling an Evaluation Team

Review how the Plus 50 data tasks fit together Customize your evaluation timeline Convene the Team for an Evaluation Launch Meeting

Reflection and Reporting (Chapter 5)

o Preliminary reporting: pulling together data with initial analysis o Reflecting on findings o Writing up memos:  Findings/Learnings  Recommendations (based on learnings and reflection) Analysis, reflection, and reporting as an iterative process

Guidance for working meetings to reflect: –With the Plus 50 Team on key findings from the process and outcome data –With the Plus 50 Team on key findings from the stakeholder feedback –With the Plus 50 Advisory Committee on all key evaluation findings Tools: –Tool 13: Evaluation Questions Tool (p. 92) –Tool 14: Qualitative Data Analysis Template (p. 95) –Tool 15: Memo Write-up Tool (including an memo outline) (p. 99) FET Tools for Reflecting on and Writing up Evaluation Findings

Working with C-PAD to Support Continuous Improvement

1.Collect Process and Outcome Data 2.Gather Feedback from Program Participants and Partners 3.Share Results with Key Stakeholders 4.Plan for Program Improvement Phase 5

Feeding two birds with one seed: Process and Outcome Data in C-PAD P&O data used for continuous improvement = P&O data used for grant-required reporting to AACC

Questions?