A joint venture between the Forum and High/Scope. Oakland Planning with Data & Kick-Off Webinar September 1, 2011 Laenne Thompson Weikart Center

© David P. Weikart Center for Youth Program Quality 2 Please press *6 to mute and un-mute. If you can hear it, so can we.

© David P. Weikart Center for Youth Program Quality 3 Please press *6 to mute and un-mute. If you can hear it, so can we. This webinar is being recorded. If you come late, leave early or just want to refresh your memory. You can access the recording afterwards.

© David P. Weikart Center for Youth Program Quality 4 Please press *6 to mute and un-mute. If you can hear it, so can we. This webinar is being recorded. If you come late, leave early or just want to refresh your memory. You can access the recording afterwards. Be Prepared: This webinar will be participatory. Be ready to be called on.

© David P. Weikart Center for Youth Program Quality 5 Agenda  Quick Introductions  Using Data to Plan & Implement Program Improvements  Improvement Planning Overview  Submitting Improvement Plans Online (tutorial)  YPQI Kick-Off  Questions and Next Steps

© David P. Weikart Center for Youth Program Quality 6 Introductions  Who’s here today?  Each site, please share:  The name of your organization  Something new you’ve tried at your site that went well since we last met

© David P. Weikart Center for Youth Program Quality 7 ASSESS PLAN IMPROVE Carry out improvement plan (Mgr coaches; Stf do) Carry out improvement plan (Mgr coaches; Stf do) Create improvement plan (YPQI Team) Create improvement plan (YPQI Team) Conduct SA (YPQI Team) [Collect other data] Conduct EA (External assessor) Youth Program Quality Improvement (YPQI) REPEAT

© David P. Weikart Center for Youth Program Quality 8 Improvement Planning Practice  As a group, we’re going to have a practice improvement planning meeting based on aggregate OFCY YPQA data.

© David P. Weikart Center for Youth Program Quality 9 Score Reports: Keep in Mind  Observation scores represent a snapshot –this has limitations and value.  The overall story is more important than the individual numbers.  What you do with it (how you step up to action) matters.  Reference: ‘How to Interpret PQA Data’ handout

© David P. Weikart Center for Youth Program Quality 10 Some pointers about YPQA data  Scores of all Fives  Not useful for identifying areas of improvement.  Extreme Variation  Might signify a misunderstanding of what the tool is measuring.  Typical trend  This distribution common as you become more comfortable with the process and the tool.  Program self assessment scores may be lower in subsequent years because raters are being more honest or more critical as they develop their reliability.

© David P. Weikart Center for Youth Program Quality 11 OFCY External Assessment* vs. National Sample** *n = all OFCY Older Youth, Wellness and Healthy Transitions and Community-Based After School programs listed in SeeChange 2010 – 2011 Interim Report, p. 14; **n = 902 sites

© David P. Weikart Center for Youth Program Quality 12 Aggregate Data Graph  Things to consider:  What does it mean?  What do you see here?  What does the data tell us?  Strengths?  Areas for growth?  What should we do as a response?

© David P. Weikart Center for Youth Program Quality 13 Aggregate Data Graph  What scores seem lower than you would want?  What item/domain do we want to focus on?

© David P. Weikart Center for Youth Program Quality 14 OFCY External Assessment* vs. National Sample** *n = all OFCY Older Youth, Wellness and Healthy Transitions and Community-Based After School programs listed in SeeChange 2010 – 2011 Interim Report, p. 14; **n = 902 sites

© David P. Weikart Center for Youth Program Quality 15 Sample Improvement Plan

© David P. Weikart Center for Youth Program Quality 16 Sample Improvement Plan

© David P. Weikart Center for Youth Program Quality 17 Sample Improvement Plan

© David P. Weikart Center for Youth Program Quality 18 Steps to make an Improvement Plan Set goal & address elements of quality (YPQA items) Implementation: Develop action steps for each goal A.Timeline B.Evaluate -(Mini) Self Assessment -Staff meeting check in -Observation-Reflection -Other? C.Discuss desired support from administrator -Training, information, etc… -Youth Work Methods courses & other further YPQA work **Don’t forget- - Involve staff in the process!

© David P. Weikart Center for Youth Program Quality 19 More to Consider: Youth Survey Data  How to use this data in your Improvement Plans?  What from the Youth Survey Data stands out as highest priorities for improvement?  What YPQA data does it correlate with?  Does the Youth Survey data help you prioritize which YPQA items to tackle in your Improvement Plan?

© David P. Weikart Center for Youth Program Quality 20 Discussion  How do you feel about this process?  Questions or potentially challenges about replicating it at your program/site?

© David P. Weikart Center for Youth Program Quality 21 Submitting Improvement Plans Online (tutorial)  You will receive your login information in an today from Leah Wallace of the Weikart Center  If you have not received an with those please contact as soon as  These logins will be used to enter both:  The Spring 2011 Improvement Plan (due 9/30)  The Fall 2011 YPQA Self-Assessment data

© David P. Weikart Center for Youth Program Quality 22 Login at scoresreporter.cypq.orgscoresreporter.cypq.org

© David P. Weikart Center for Youth Program Quality 23 Why do I need to enter my plans?  It allows you to track your goals and your progress in reaching those goals from year to year.  It allows your program leader to tailor professional development opportunities  It helps create an overall picture of what organizations in your city are prioritizing to improve the quality of their programs.

© David P. Weikart Center for Youth Program Quality 24 YPQI TIMELINE

© David P. Weikart Center for Youth Program Quality 25 Next Steps & Expectations  Meet with your program team and choose goals for your Improvement Plan  Determine actions steps and monitoring activities for each goal  Finalize your Improvement Plan: enter online by September 30

© David P. Weikart Center for Youth Program Quality – 2012 OFCY Grantee Youth Program Quality Improvement Kick-Off

© David P. Weikart Center for Youth Program Quality 27 YPQI Kick-Off:  Review: What is the YPQA?  What is the YPQI?  Self-Assessment  Next steps and Key Dates  Q/A

© David P. Weikart Center for Youth Program Quality 28 Transfer: Application of skills in new settings Time: Multiple sessions Setting: Point of service Quality of Instruction & Content Quality of Instruction & Content Youth Engagement Proximal skill building Social emotional 21 st Century Cognitive Proximal skill building Social emotional 21 st Century Cognitive Short-term outcomes Achievement Healthy behavior School retention Short-term outcomes Achievement Healthy behavior School retention Quality Matters

© David P. Weikart Center for Youth Program Quality 29 YOUTH PROGRAM QUALITY ASSEMENT (YPQA)

© David P. Weikart Center for Youth Program Quality 30 What is the YPQA? The Youth PQA is a research validated instrument designed to assess the quality of youth programs for the purposes of accountability, evaluation, and program improvement. The instrument has been used in a wide variety of settings including after-school, community-based, camp, drop-in, and mentoring.

© David P. Weikart Center for Youth Program Quality 31 The Pyramid of Program Quality  Youth are emotionally and physically safe.  Youth are supported by staff and engaged in hands-on skill building  Youth work together and experience a sense of belonging  Youth have access to high level developmental experiences: planning, setting, goals, making choices and reflecting Plan Make choices Reflect Partner with adults Lead and mentor Be in small groups Experience belonging Reframing conflictEncouragement Skill building Active engagement Session flow Welcoming atmosphere Healthy food and drinks Program space and furnitureEmergency procedures Psychological and emotional safety Physically safe environment

© David P. Weikart Center for Youth Program Quality 32 The YPQA is…  …rooted in idea of Positive Youth Development  …content-neutral  …a measurement that guides learning and professional development  …a tool to measure what happens at the point-of- service, where staff and youth interact

© David P. Weikart Center for Youth Program Quality 33 Higher scores at higher domains are associated with higher levels of youth engagement  Youth self-reports of:  Belonging  Interest  Challenge  Learning

© David P. Weikart Center for Youth Program Quality 34 High quality instruction provides youth with opportunities to practice emerging social and emotional skills…  Efficacy  Communication  Empathy  Problem Solving … that support success in adolescence and early adulthood.

© David P. Weikart Center for Youth Program Quality 35 Lower scores are associated with youth disinterest

© David P. Weikart Center for Youth Program Quality 36 WHY THE YPQI? YOUTH PROGRAM QUALITY IMPROVEMENT

© David P. Weikart Center for Youth Program Quality 37 Theory of Change: Multiple Levels of Settings Point of Service Setting Quality Instruction & Proximal Child Outcomes This is the youth at your programs

© David P. Weikart Center for Youth Program Quality 38 Theory of Change: Multiple Levels of Settings Organizational Setting Continuous Improvement Practices for Site Teams This is your organization and staff

© David P. Weikart Center for Youth Program Quality 39 Theory of Change: Multiple Levels of Settings Point of Service Setting Organizational Setting What you do here…. Will improve the experiences for kids here

© David P. Weikart Center for Youth Program Quality 40 Benefits of YPQI  Creates a common language of quality  Emphasizes positive youth development  Provides a framework for professional development

© David P. Weikart Center for Youth Program Quality 41 SELF-ASSESSMENT AND THE YPQI What’s new for ?

© David P. Weikart Center for Youth Program Quality 42 This is the PQA Self Assessment Process ASSESS Standardized Assessment of Instruction PLAN Team-based Planning with Data IMPROVE Coaching and Performance Feedback IMPROVE Training for Instructional Skills

© David P. Weikart Center for Youth Program Quality 43 Assess The PQA allows you to observe what actually goes on in your youth programs? How often do you have the time or energy to simply observe a program led by another staff? How often do you rely on what you think happens in your programs?

© David P. Weikart Center for Youth Program Quality 44 What does the Assess Stage look like?  Team Selection  Team Training  Planning the Data Collection  Collecting Data  Team Scoring Meeting

© David P. Weikart Center for Youth Program Quality 45 Team Selection  Select 1-3 people from your organization to be on the self-assessment team  Make sure to include frontline staff  Attend a Youth PQA Basics Training

© David P. Weikart Center for Youth Program Quality 46 Observations and Consensus  Plan to observe 2-4 program offerings  Observe at least minutes, but try to observe the entire offering  Come together with all your data and come to consensus scores for the entire program based on that data.  The conversation is the most important part of this process !

© David P. Weikart Center for Youth Program Quality 47 Data Entry You will enter your final scores into an “online scores reporter” data base. This will allow you to make reports and compare your data form year to year.

© David P. Weikart Center for Youth Program Quality 48 Plan Looking at the results of your assessment gives you a chance to make concrete plans. What are your program strengths? What do you want to improve?

© David P. Weikart Center for Youth Program Quality 49 Planning with Data This workshop will allow you to look at your assessment data in depth and create improvement plans based on it.

© David P. Weikart Center for Youth Program Quality 50 Improve During the improve phase you will implement your plans. What kind of professional development do you need? What additional resources? What changes to the curriculum?

A joint venture between the Forum and High/Scope. IMPROVE: Methods Workshops This improvement process is supported with professional development workshops Aligned Youth Work Methods Courses Voice & Choice (V-C, V-D, III-N) Planning & Reflection (IV-P, IV-R) Building Community (III-L) Cooperative Learning (III-M) Active Learning (II-H)Scaffolding for Success (II-I) Ask-Listen-Encourage (II-J)Reframing Conflict (II-K) Structure and Clear Limits Program Component: Homework Help These 2- to 3-hour workshops are aligned with the items of the Youth PQA found in the pyramid. Workshops may take place for a network or a program. Safe Environment Supportive Environment Interaction Engagement

© David P. Weikart Center for Youth Program Quality 52 NEXT STEPS

© David P. Weikart Center for Youth Program Quality 53 YPQI TIMELINE

© David P. Weikart Center for Youth Program Quality 54 Any Questions?