1. A joint venture between the Forum and High/Scope. YPQA Youth Assessor Webinar YPQA Youth Assessor Webinar Cleveland Orange County, NY December 18 th, 2009

2. © David P. Weikart Center for Youth Program Quality 2 Agenda  Introductions  Cleveland Presentation  Orange CO. Presentation  Summary and Implications  Next Steps

3. © David P. Weikart Center for Youth Program Quality 3 Youth Engagement in Community Change

4. © David P. Weikart Center for Youth Program Quality 4 8 Principals of Youth Engagement

5. Q-Team Cleveland, OH

6. Purposes Purposes Q. Why did you pursue working with youth as assessors? What are their goals?Q. Why did you pursue working with youth as assessors? What are their goals? A. In 2006 Ralph Pitman (Associate Director of START, Lutheran Metro. Ministry Youth Services) completed the YPQA Intermediate training. Following this we brought Monica Jones to Cleveland to do an assessment of our after school program. We then hired an intern from Oberlin College, trained her in YPQA and did a round of internal assessments to test the correlation between external and internal assessments. Based on the comparable results we decided that the next step would be to take the Program Quality Pyramid seriously and engage youth program participants in YPQA training. Our goal is to create a self sustaining program of peer to peer program quality assessment and improvement.A. In 2006 Ralph Pitman (Associate Director of START, Lutheran Metro. Ministry Youth Services) completed the YPQA Intermediate training. Following this we brought Monica Jones to Cleveland to do an assessment of our after school program. We then hired an intern from Oberlin College, trained her in YPQA and did a round of internal assessments to test the correlation between external and internal assessments. Based on the comparable results we decided that the next step would be to take the Program Quality Pyramid seriously and engage youth program participants in YPQA training. Our goal is to create a self sustaining program of peer to peer program quality assessment and improvement.

7. How? Q.How do they fit into the larger Quality Improvement process? How did you adapt the standard YPQA process for youth Assessors? How did you prepare them? Are there any best practices that you would like to share?

8. Cont. A. The after school program Director selected 8 teens who had exhibited leadership in their program through regular attendance over a number of years, participation in small groups, positive attitude, and participation in the program Advisory Committee. These teens met monthly for two hours for a year with Pitman and were later joined by Pat Heilbron (Director, Treu Mart Fellowship at Case Western Reserve University). The “curriculum” was excerpted from the YPQA training and the T.O.T. training materials. An emphasis was placed on teaching techniques for observation, since it was believed that this was the most difficult for anyone to learn and that the theoretical framework could be taught later once the group learned to make reliable observations, and how score the YPQA. At this point we observe in groups of at least two and then we score as a larger group challenging each other and working to reach agreement on the final scores.A. The after school program Director selected 8 teens who had exhibited leadership in their program through regular attendance over a number of years, participation in small groups, positive attitude, and participation in the program Advisory Committee. These teens met monthly for two hours for a year with Pitman and were later joined by Pat Heilbron (Director, Treu Mart Fellowship at Case Western Reserve University). The “curriculum” was excerpted from the YPQA training and the T.O.T. training materials. An emphasis was placed on teaching techniques for observation, since it was believed that this was the most difficult for anyone to learn and that the theoretical framework could be taught later once the group learned to make reliable observations, and how score the YPQA. At this point we observe in groups of at least two and then we score as a larger group challenging each other and working to reach agreement on the final scores.

9. Success Stories and Outcomes Q. How has it worked? Are there any stories to share? What have been the benefits for the young people and the programs that they are assessing/working with?Q. How has it worked? Are there any stories to share? What have been the benefits for the young people and the programs that they are assessing/working with?

10. Cont. A. Using our own programs as “sheltered” learning settings for the Q Team we did observations and scored the YPQA in three different program settings. In each one we learned new lessons. In one, an outside neighborhood clean up we walked around with the participants and learned how difficult it was to observe activities that are spread out over a large area. We also learned that when there are conflicts between adults in the program, the presence of youth observers represents a threat. In another program we learned that youth from a new neighborhood may initially be reserved when being observed, they soon learn to ignore the observers as long as their role is clearly explained. We have come to appreciate that the youth observations are becoming increasingly reliable, and that they often observe details that an adult observer might miss. We were surprised and delighted to find one youth using all his senses to make program observations!A. Using our own programs as “sheltered” learning settings for the Q Team we did observations and scored the YPQA in three different program settings. In each one we learned new lessons. In one, an outside neighborhood clean up we walked around with the participants and learned how difficult it was to observe activities that are spread out over a large area. We also learned that when there are conflicts between adults in the program, the presence of youth observers represents a threat. In another program we learned that youth from a new neighborhood may initially be reserved when being observed, they soon learn to ignore the observers as long as their role is clearly explained. We have come to appreciate that the youth observations are becoming increasingly reliable, and that they often observe details that an adult observer might miss. We were surprised and delighted to find one youth using all his senses to make program observations! The benefits for the young people are noticeable. The group is engaged in meetings now, after almost 2 years. They are taking increasing responsibility for being on time and prepared for meetings and public appearances. They are appropriately reserved, but not shy when asked to interact with adults, and are gaining confidence with each new experience.The benefits for the young people are noticeable. The group is engaged in meetings now, after almost 2 years. They are taking increasing responsibility for being on time and prepared for meetings and public appearances. They are appropriately reserved, but not shy when asked to interact with adults, and are gaining confidence with each new experience.

11. Challenges Challenges Q. Have there been any? How have they been addressed?Q. Have there been any? How have they been addressed? A. There are many challenges.A. There are many challenges. 1. Now that we have gained sufficient experience for the youth to have a sense of what they know and what their limitations are we all need to agree on a process of how to take our learning to a higher level so they can function in the community in a truly professional manner.1. Now that we have gained sufficient experience for the youth to have a sense of what they know and what their limitations are we all need to agree on a process of how to take our learning to a higher level so they can function in the community in a truly professional manner. 2. This increased training and the increased public demand for their services will demand greater amounts of their time 2. This increased training and the increased public demand for their services will demand greater amounts of their time 3. Our youth come from low income background as they get older they have increasing responsibilities that require their time for education and careers.3. Our youth come from low income background as they get older they have increasing responsibilities that require their time for education and careers.

12. Challenges Continued 4. It is a goal and a vision that the Q Team will become a self supporting service that will employ youth members at a reasonable rate for their services.4. It is a goal and a vision that the Q Team will become a self supporting service that will employ youth members at a reasonable rate for their services. 5. We struggle to find the right balance between adult support and leadership in the community and youth leadership and engagement. Our vision is that this will be a fully youth-lead service with adults in the role of consultants and facilitators. This will require careful planning and scaffolding.5. We struggle to find the right balance between adult support and leadership in the community and youth leadership and engagement. Our vision is that this will be a fully youth-lead service with adults in the role of consultants and facilitators. This will require careful planning and scaffolding.

13. Next steps Q. What are your goals and next steps moving forward?Q. What are your goals and next steps moving forward? A. a. Secure youth by-in for additional trainingA. a. Secure youth by-in for additional training b. Develop community relationshipsb. Develop community relationships c. Secure funding partners for next stage in Q Team Developmentc. Secure funding partners for next stage in Q Team Development d. Develop strategic plan for Q Team to include nationally recognized training components, participation of other youth in the community, finding and capacity building.d. Develop strategic plan for Q Team to include nationally recognized training components, participation of other youth in the community, finding and capacity building.

14. Orange County : Incorporating Youth Assessors The purpose of using the YPQA is to assess the quality of Youth Programs. Done through observation by trained external assessors this instrument is great in observing the process, as well as the inputs and outputs of the youth program. ~ HOWEVER ~ With that said we were still left to wonder, if the assessment instrument is used to assess youth program quality, who better to get feedback from than YOUTH THEMSELVES???

15. 1) Dissected instrument and identified subscales that interests us most as Youth Program Funders 2) Dissected desired subscales and identified which items and indicators interest us most as Youth Program Funders 3) Presented identified YPQA subscales to college level intern to collect feedback. 4) Formatted desired subscales to fit on one page and modified language to appropriate age level 5) Staff and college intern create a Youth Assessor Curriculum 6) Practice and review with additional staff 7) Youth Assessor training for High School level summer interns (implemented by College level intern) 8) Youth implement assessments of summer programs 9) Review and debriefing (post-assessment) Steps Orange County used to Incorporate Youth Assessors

16. Youth Participation as YPQA Assessors Increases: Youth Voice Youth Leadership Opportunities Youth Input Youth belongingness to the community or their personal program Assessment Quality Assessment Diversity (providing feedback from different age groups)

17. Orange County : Challenges Validity of the instrument Helping youth to be objective vs. subjective in observations and reporting

18. Orange County : Next Steps Connecting with other locations using the youth assessors. Have youth assessors observing every agency once a year Validating the youth facilitator version of the YPQA instrument !!!

19. © David P. Weikart Center for Youth Program Quality 19  Lessons learned?  Challenges and Questions about the Project  Implications for future work Summary

20. © David P. Weikart Center for Youth Program Quality 20 Discussion

21. © David P. Weikart Center for Youth Program Quality 21 Resource: forumfyi.org/files/FINALYouth_Engagment_8.15pdf.pdf forumfyi.org/files/FINALYouth_Engagment_8.15pdf.pdf