1 Quality Matters: Building Capacity and Investment in Youth Program Quality The Center for Youth Program Quality

2 Objectives Understand how quality defined and measured by the Youth Program Quality Assessment and other quality assessment tools Learn about quality improvement systems currently implemented in numerous statewide and place-based networks

3 Systems for Quality Leveraging existing “change” resources POS Quality Access to key developmental and learning experiences Professional Development Local Evaluators Accountability Measurement

4 At best, schools fill only a portion of developmental “white space.” Who fills the rest? And what is the “locally appropriate mechanism for monitoring the availability, accessibility, and quality of programs…” in school and out? Saturating communities with “ample programs” requires improving the quality and reach of all the systems, settings and programs that touch young people’s lives. Selective replication and improvement are not sufficient. school Ages Times of Day/Year Outcome Areas after- school Developmental “White Space” Quality & Reach… Looking into the developmental white space

5 Children’s Services in LA County SOURCE: Margaret Dunkle The Landscape of Youth Programs

6 Capacity to Recruit, Train, Retain Workforce Cross-system convening/coordination mechanisms Widely adopted assessment and monitoring procedures Buy in re definitions, quality standards, accountability requirements Strong Policy / Leadership Horsepower Decision-maker engagement & coordinating structures Aligned policies for quality accountability and improvement Shared vision, strong demand, active family/youth involvement Capacity to Assess & Improve Programs Professional development opportunities/incentives Adequate assessment and improvement training/TA capacity Strong, Stable Program Base Healthy program landscape (distribution & focus) Accurate data on workforce (skills, supports, recruitment, retention) Cross-system provider networks and communications Domains Elements Cross-system program data base/info source The Ready by 21™ Quality Counts Framework

7 What is program quality? ??? inputs outcomes youth program Another way to say it: What do we want to see in high quality youth programs?

Defining Quality Several ways to organize: Readin’ + ‘Ritin + ‘Rithmatic (old-school) Affect + Active Learning + Metacognition (Education) Relatedness + Autonomy + Competence (Psychology SDT) ContentTherapeutic process What kids need… Relationship + Task + Increasing Complexity What adults should do…

9 Our Quality Construct: The Pyramid of Program Quality Plan Make choices Reflect Partner with adults Lead and mentor Be in small groups Experience belonging Engagement Reframing conflictEncouragement Skill building Active engagement Session flow Welcoming atmosphere Supportive Environment Interaction Safe Environment Healthy food and drinks Program space and furnitureEmergency procedures Psychological and emotional safety Physically safe environment

10 Viewing Quality in a Systems Context SAE System Accountability Environment PLC Professional Learning Community POS Point Of Service Safety Support Interaction Engage

11 Defining Quality What can management do? Learning Features Community Continuity Performance Feedback Data & Information External monitoring Lesson & curriculum review Daily supervision & support by site coordinator Pre-session planning Frequent staff meetings Management Priorities/Values Youth engagement & content relevance Purposeful relationships Youth voice structures Focusing Features

12 Engagement Interaction Supportive Environment Safe Environment Professional Learning Community 6 standards 4 standards 33 standards 19 standards 1 standard Other 32 standards 49 standards Youth Organizational Voice Source: (1998). The NSACA Standards for Quality School-Age Care. There are thirty-six keys of quality and 144 total standards Defining quality at the system level Current Regulatory Models Miss the POS

13 The Youth Program Quality Assessment (YPQA) –Validated observational assessment tool –Measures quality at the point of service –Assesses frequency and access to key developmental experiences –Can be used to assess progress over time –Content-neutral for use across settings, ages, systems

14 Form B Organizational Interview Ask questions, write, score (2 hours) Form A Observation Watch, write, score (3 hours) Organization Program Offering 1 Program Offering 2 Program Offering 3 Program Offering 4 Structure of the Youth PQA

15 Sample Item from the Youth PQA III. Interaction III-L. Youth have opportunities to develop a sense of belonging. Note: Structured refers to the quality of being intentional, planned, and/or named; it does not refer to informal conversation. IndicatorsSupporting Evidence 1Youth have no opportunities to get to know each other (beyond self-selected pairs or small cliques). 3Youth have informal opportunities to get to know each other (e.g., youth engage in informal conversations before, during, or after session. 5Youth have structured opportunities to get to know each other (e.g., there are team-building activities, introductions, personal updates, welcomes of new group members, icebreakers, and a variety of groupings for activities) The Youth PQA consists of 7 Domains (4 in A, 3 in B) 30 Scales (18 in A, 12 in B); 103 item rows (60 in A, 43 in B) “Domain” “scale” “item row”

What POS Quality Looks Like on the Ground Occurred For All Sample of nearly 600 different youth workers Parallel findings in schools research Occurred For Some Did Not Occur 8/13/200816The Center for Youth Program Quality

Does it Work? Findings from Several Samples POS quality-outcomes findings: –Supportive environment related to: Attendance –Interaction related to: Interest in program –Engagement related to: Sense of challenge, sense of growth, school-day reading, school-day suspension –Note: No offerings get to high engagement without high support and high interaction Quality Improvement (YPQI) Findings –Scores increase from pre to post –Scores increase in the targeted areas more –Management practices are related to quality change (Vision, Feedback, Continuity)

How we think about DDCI - People change not programs Contemplation Preparation Action Maintenance Quality assessment Planning with data Implementation & coaching Repeat cycle SAE System Accountability Environment PLC Professional Learning Community POS Point Of Service Individual Change Model Organizational Context Prochaska, J.O., & DiClemente, C.C. (1982). Transtheoretical therapy toward a more integrative model of change. Psychotherapy: Theory, Research and Practice, 19(3), /13/200818The Center for Youth Program Quality

Competencies for Youth Development Practitioners Emergence of POS quality culture Spend time planning & prepping activities Improve practice systematically "Team focused on youth experience" mentality Intentionality in POS quality practice Icebreakers and inclusion Cooperative grouping strategies Targeting learning edge – scaffolding, engaging youth in planning & reflection, active learning Providing opportunities for youth voice and leadership 19

Competencies for Management –Knowledge/understanding of positive youth development methods individually and integrated as the High/Scope participatory learning approach; –Conduct reliable performance assessment based on observation and validated measurement rubrics; –Conduct performance coaching based on strengths based transmission of performance data; –Lead a staff team through a data driven quality improvement process 20

System Level Outputs –Accountability policy with high returns in staff buy-in and learning; –Common language and terminology supporting focus on quality at the point of service; –An integrated professional development investment that is integrated over time as a sequence of PD experiences, integrated across levels of organization and professional roles, and integrated with ultimate program purposes in positive youth development; 21

Outcomes –High quality programs –Better staff retention and retention of better staff –Greater impact on youth development and learning 22

23 mbus etroit Minneapolis Kentucky Iowa Oklahoma New York Rhode Island Austin Georgetown Divide Columbus Indianapolis Grand Rapids Nashville St. Louis Washington* West Palm Beach County Rochester Chicago Systems for Quality Accountability Policies in Places YPQA is part of state and county accountability policies: –Cross sector (DHS& DOE) snapshots: Iowa, Washington, Arkansas –Statewide 21st Century: Michigan, Maine, Minnesota, Rhode Island, New Mexico, –Cities and Counties: Rochester, Detroit, Grand Rapids, Palm Beach

24 Designing Quality Improvement Systems (QIS) System Capacity Program Staff Skill & Knowl Self- assessment of Quality (b) Planning with Data (c) External quality assessment (a) Quality Advisor (e) TOTs for quality assess, coaching, and youth work methods (f,g) Coaching & Training Phase 1: Readiness & Capacity Phase 2: Impact & Sustainability Targeted youth work methods training for direct staff (h) Quality coaching by managers (i) Use of on-line dashboards and training (d) External Quality Report with Norms Self-Assessed Quality Report Use of on-line dashboards and training (d) Mostly Mangers Managers and direct staff

25 Defining the Purpose of Your QIS Lower Stakes Program Self-Assessment Rough data to get staff thinking and discussing program quality in the context of best practice Less time Less money Impact internal audiences Higher Stakes External Assessment Precise data for internal and external audiences for evaluation, monitoring, accountability, improvement, reporting More time More money Impact internal and external audiences (the creative middle)

Columbus Indiana Phase 1: Building Local Capacity STEP 1 Decide to build system STEP 2a Self- assessment STEP 2b External assessment STEP 3 Plan for improvement STEP 4 Carry out plan SAE System Accountability Environment PLC Professional Learning Community August 25 Quality Matters Presentation August 26 Youth PQA Basics January Ext Assessment October 8 Planning with Data Improvement Plan Opt Phase 2 Method Workshops Quality Coaching STEP 5 Measure change Annually Program SA Ext Assessment Observe-Reflection Planning with Data POS Point Of Service

Questions about… Purpose? Process? Pilot? Next Steps… 27