CVPP Evaluation: Youth Employment Program & Parent Program Presentation by: Jessica Reichert, Senior Research Analyst Illinois Criminal Justice Information Authority CVPP Steering Committee Meeting February 13, 2014

EVALUATION OF YOUTH EMPLOYMENT PROGRAM

YEP evaluation methods Administrative data Job readiness training survey Mentor training survey Employer exit survey Coordinator manager survey Mentor exit survey Youth exit survey Youth pre- and post-assessment

Lead Agencies: Program data 7,254 youth applied to the program 1,894 were accepted 1,604 completed job readiness training 1,838 were assigned a mentor 1,466 completed employment

Youth: On job readiness training Agreed or strongly agreed about the training: Aspect of trainingPercent Trainer knowledgeable, helpful84% Offered better sense how to obtain, maintain a job 82% Answered questions about employment80% Materials, handouts useful79% Well-designed72%

Youth: On job readiness training Spend more time on Resumes, applications (n=48) Interview techniques (n=44) Money management (n=29) Spend less time on: Dressing appropriately for a job (n=45) Hygiene (n=31)

Youth: On job readiness training Liked best about the training: Learning job readiness/life skills (n=152) Activities/role plays/games (n=95) Teamwork/ group discussion (n=81) Change about the training Nothing (n=228) More interactive activities (n=131) More organized/prepared (n=69)

Mentors: On training Agreed or strongly agreed about the training: Aspect of trainingPercent Well designed88% Information about mentoring89% Materials, handouts useful89% Trainer knowledgeable, helpful91% Learned what it takes to be a mentor84%

Mentors: On training Spend more time on: Nothing (n=31) More time with youth (n=13) Dealing with problems, crises, emergencies (n=10)

Employer: Program ratings Ratings: YEP was successful or very succesful-85% Satisfied or very satisfied with their experience as a YEP employer- 84% Satisfied or very satisfied with matching of youth with their agency- 81% Good or very good communication with YEP staff - 80% YEP youth were prepared or very prepared 71%

Employer: Interested in serving as YEP employer again

Employer: Hiring YEP youth

Employer: Change about program Nothing (n=15) More preparation for youth (n=15) Longer program/more hours for youth (n=13) Better communication with program (n=12)

Coordinators/Managers: Ratings Aspect of programPercent Employment component84% ICJIA support75% Lead agency support75% Mentor component68% Training50% Service and support from CAPs24% Rated aspects of YEP good or very good:

Coordinator/Manager: Comment “I really appreciated the fact that it kept a lot of kids off the street this summer and gave them a chance, not only to do something positive, but earn money.”

Mentors: Ratings Aspect of programPercent Mentor component86% Employment component84% Manager/Coordinator support74% Mentor training68% Rated aspects of YEP good or very good:

Mentor: On relationship w/ mentee Aspect of relationshipPercent Made a positive connection with their mentees 83% Interested in serving as a mentor again82% Found it easy to be a mentor63% Learned new things about themselves58% Rated aspects of YEP good or very good:

Youth: How learned about YEP How learned about YEP

Youth: Ratings of program Aspect of programPercent Mentor91% Job tasks89% Job readiness training88% YEP overall87% Supervision on the job86% Rated aspects of YEP good or excellent:

Youth: Job training prepare you for job?

Youth: Training skills used Time management- 77% Dressing appropriately for the job- 75% Professional vocab, communication- 70% Money management- 59% Conflict resolution- 54%

Youth: Job a good match?

Youth: Work benefitted employer?

Youth: Offered a job?

Youth: Gain from mentoring? Most common responses: Guidance/general advice (n=85) A Relationship/Someone I can talk to and trust (n=82) Confidence/self-esteem (n=62) Communication skills (n=52)

Pre- and post-assessment Measured: attitudes toward employment attitudes toward violence conflict resolution self-esteem Received: 2,068 forms 1,446 pre- and 622 post-assessments 368 matched pre- and post-assessments

Pre- and post- results nMean 1 (pre) Mean 2 (post) Change in means Attitudes toward employment Attitudes toward violence* Conflict resolution* Self-esteem Combined measures

Conclusions from YEP evaluation Youth very satisfied with program, job training, job, and mentor Small decreases in some mean pre- and post- scores (started and ended high) Mentors rated training and program high Recommendations to improve payroll Employers satisfied with program, some suggestions to better prepare youth

Suggestions to enhance YEP Focus on youth at-risk, in need Selection process – Limit college students – Limit prior participants – Priority to those with jj or cj involvement – Priority to those with no prior employment – Priority to those with limited, no opportunities for another job

Suggestions to enhance YEP Encourage education for higher level, higher paying jobs Enhance mentoring component Mentors should discuss, help mentees with – conflict resolution – self-esteem – violence

Suggestions to enhance YEP Enhance evaluation – Improve survey administration, completion rate – Repeated instruction by Evaluator – Encourage administer the same way each time, no distractions, enough time allotted – Add questions

EVALUATION OF PARENT PROGRAM

Evaluation of PP methods Administrative data Parent Leader training evaluation Coordinator/ Manager exit survey Parent Leader exit survey Parent Leader pre- and post-survey

Lead Agencies: Program data 983 Parent Leaders recruited 834 Parent Leaders trained 4,816 hours spent on service projects 107 projects completed

Parent Leader: Training ratings Aspect of trainingPercent Well designed94% Trainers knowledgeable96% Materials, handouts useful95% Protective factors covered96% Agreed or strongly agreed:

Parent Leader: Learned from training Learned at trainingPercent How to strengthen own family95% How to strengthen community90% How to form parent teams89% How to create service teams87% Agreed or strongly agreed:

Coordinators/Managers: On program Aspect of programPercent Support from ICJIA75% Support from BSF90% Preparation for role88% Training for role86% Support from Lead Agency71% Rated good or very good:

Coordinators/Managers: Community service Improved communityIncreased protective factors

Coordinators/Managers: Continuing service project

Parent Leaders: On program Aspect of programPercent Support from Manager/Coordinator96% Training for your role92% Resources to complete service projects85% Successfulness of program78% Rated good or very good:

Parent Leaders: Incorporate what learned in daily life?

Parent Leaders: Community service Improved community Increased protective factors

Pre- and post- survey Measured: family functioning and resiliency social and concrete support nurturing and attachment Child development/ knowledge of parenting Received: 889 surveys 613 pre- and 276 post-surveys 204 matched pre- and post-surveys

Pre- and post- results nMean 1 (pre) Mean 2 (post) Change in means Family functioning and resiliency* Social and concrete support Nurturing and attachment Child development and knowledge of parenting Combined measures*

Conclusions from Parent Program evaluation Program increased protective factors Parent Leader trainings well received Administrative teams’ responses were favorable to the program Parent Leaders thought program was well conducted and successful

Suggestions to enhance PP Increase protective factor of social and concrete support Reduced change in means on 3 items: – I have others who will listen when I need to talk about my problems. – I wouldn’t know where to go for help if I had trouble making ends meet. (reverse coded) – If I needed help finding a job, I wouldn’t know where to go for help.

Suggestions to enhance PP Recruit younger parents with young children – More at-risk for child abuse, neglect Continue to recruit primary caregivers – To have greatest impact on parenting; training material most relevant to them Recruit more fathers – Involvement in child’s life is a protective factor, can reduce child maltreatment

Suggestions to enhance PP Enhance evaluation Improve survey administration, completion rate Add questions: – Parent Leaders: Age; gender; number & ages of children; status as primary caregiver of own children or grandchildren; loss of custody of children, prior DCFS involvement – Service projects: Number of participants/attendance per project; duration of service project

Thank you! Questions/comments Reports will be available Spring 2014 on website