1. Engaging Youth in Process and Outcomes Evaluations of an Out-of-School-Time Program Amy Corron, United Way of Greater Houston Roger Durand, University of Houston-Clear Lake** Emily Gesing, United Way of Greater Houston Julie Johnson, Communities in Schools of Houston, Inc. Kevin Kebede, Alief YMCA Jennifer Key, Alief Independent School District Linda Lykos, YMCA of Greater Houston Cheryl McCallum, Children’s Museum of Houston ** Presenter

2. Outline of the presentation…..  Background  Importance and purpose  Means of engaging youth in evaluation ….  Results of involving youth  Problems and prospects of engaging youth  Discussion: Youth empowerment  Some useful, important references

3. Background  The “Houston’s Kids Program,” an out-of-school time program, was created in the aftermath of Hurricanes Katrina and Rita  Objective: to address the needs of displaced and other at- risk youth (grades 5-8 and high school age youth)  Collaborating partners: The United Way of Greater Houston; Communities-in-Schools of Houston; the YMCA of Greater Houston; the Alief YMCA; the Alief Independent School District; and the Children’s Museum of Houston.  “Seed money”: America’s Promise Foundation

4. Importance and purpose  We seek to extend previous work on youth participatory evaluation and as well as on empowerment evaluation  Our work also seeks to bridge the “digital divide” and to enlighten about youth as agents of community change  Our intention is to provide rich, in-depth information about youth functioning as evaluators as well as about the shortcomings and strengths of activities intended to develop critical assets in youth (Search Institute).

5. Means of engaging youth in evaluation…  Toddler focus groups (led by group counselors)  Projective device for toddlers: drawings about family and friends  Youth surveys (grades 4 through 8 and high schoolers)  “Design your own program activity” (grades 4 though 8)  “Sticker surveys” (grades k-3)  Involving youth with the program committee

6. Results of involving youth….(what worked well)  Design your own program activity…resulted in changes in program activities, including more emphasis on science technology and discussions about bullying; removing a “teen talk activity.”  Youth surveys….  Grades 4 to 8 led to understanding of program impacts  High school: led to improving some activities (e.g., resume writing)  Sticker surveys …impacts on toddlers

7. Results of involving youth….(what did not work so well)  Projective drawings by toddlers of family and friends  Involving youth with the program committee

8. Problems and prospects of engaging youth….(Principal problems)  Toddler focus groups…counselors may have “read in” too much; group reporting  Project drawings by toddlers …difficult to interpret; problems in reliability  Surveys of youth in grades 4 though 8…difficult to get answers to open-ended questions  Design your own program activity ….lack of maturity in youth  Sticker surveys of toddlers….could only ask simple “yes-no” questions  Involving youth on committees….motivation!!  Limits on the use of statistical models and inferences… measurement levels

9. Problems and prospects of engaging youth….(Principal prospects)  Triangulating multiple measurement results from different data collection methods produced rich, useful evaluation findings  Projective drawings produced some disturbing findings about family problems  Youth surveys led to charting the growth in developmental assets attributable to the program  “Design your own program activity” led to important program additions (e.g., more science technology emphasis, discussions of bullying) and “deletions” (e.g., teen talk) desired by program participants  Sticker surveys of toddlers provided direct, candid answers

10. Discussion: Youth empowerment  Engaging youth as part of a strategy of client-centered evaluations of out-of-school-time programs  Youth empowerment and its limits: The need to balance the interests of clients (youth) and other stakeholders (teachers, the district, funders)  Where does youth empowerment fall in relation to insuring fidelity to an evaluation’s purpose? Is there a conflict involving AEA Guiding Principles?  The need and difficulties of preparing youth for/in evaluation  Engaging youth in evaluation and asset development: it’s therapeutic for youth  Listening to what youth have to say: its also therapeutic for adult stakeholders

11. Some useful, important references….  Kim Sabo Flores, Youth Participatory Evaluation: Strategies for Engaging Young People. San Francisco: Jossey-Bass, 2008.  David Fetterman, Empowerment Evaluation in the Digital Villages: Hewlett- Packard’s $15 Mission Race Toward Social Justice. Stanford University Press, 2013.  Jennnifer Gong and Dana Wright, The Context of Power: Young People as Evaluators, American Journal of Evaluation, September 2007; vol. 28, 3: pp. 327-333