Guidance for Completing Interim Report I Evaluation Webinar Series 3 Dec 2013

Housekeeping Online Tool Available Now! Webinar Format Review Questions / Provide Guidance Tips & Additional Assistance Town Halls (Open Q & A Forum) Webinars (Evaluators & TX21st) ACE Help Desk

Texas ACE Evaluation Consultant Mary Ann Spracher

Component 2: Implementation Practices ResourcesImplementationOutputs - ActivitiesOutputs - ParticipationIntermediate OutcomesImpact HUMAN (All staff and others that work in your program - include names of key personnel) Include: # of students and families you propose to serve SUPPORT (Partners, vendors, stakeholders, funders, and so forth) CURRICULUM (Be specific: Name the curriculum) OTHER? Special kinds of facilities, field trips that are related to program activities and outcomes School Program Alignment (How is your program specifically aligned with the regular school day. Recruiting and Retaining (right students, right mix of students) (How do you recruit & retain students/families and how do you determine which to recruit? What systems, strategies, processes, people, and tools do you currently use?) Integrating Student & Family Voice (How do you include student / family voice & choice at the macro (center) level and at the micro (activity) level?) Ongoing Monitoring (data use and observation) (How do you currently monitor your program? What tools do you use? How often) Professional Development (What kinds of professional development do you and your staff participate in & how often?) Academic Support Name of activity - brief description of what you are working on. Enrichment Name of activity - brief description of what you are working on. Family Engagement Name of activity - brief description of what you are working on. College and Career Name of activity - brief description of what you are working on. Activity name, targeted number of students, & instructor (if certified teacher or other please indicate here). List the total number of days & hours for the semester for each activity (i.e. Homework Help students, 70 days, 70 hours) Increased Attendance Increased Academic Performance Decrease Problem Behavior Increase Promotion Rates Increase Graduation Rates Anything else you currently measure would go here as well EXAMPLES: Increased student engagement, family engagement, increased focus of control, school bonding, family bonding and so forth depending upon what you are wanting to achieve. ALL STUDENTS GRADUATE READY FOR COLLEGE AND CAREER

Evidence At a minimum, these Implementation Practices include evidence of the following 5 essential components: School Program Alignment Recruiting and Retaining participants (Students and their families are recruited based on at-risk profile – and we know how and why we are recruiting the “right” kids and “right mix” of kids) Integrating student and family voice and choice Ongoing Monitoring (data use and observation) Professional Development for program personnel

Your ACE Program The intention of this this exercise is to provide you with a current snapshot of your implementation practices. Once thoroughly complete, it will provide you with insight that will enable you to address gaps in your program. By responding YES to any question in this report, you are saying that you can produce clear evidence that these practices are currently in use today. Answering NO to a question does not mean that your program is in jeopardy, but rather that there are elements of your current implementation practices that are evolving.

Completing the Report The best way to complete this report, is to refer to column 2 of your center logic model. Remember, the information is only relevant to your program if the practices you have in place in your prime blueprint are actualized into the everyday practices at the center.

School Program Alignment The questions of this essential component of Implementation Practices are asking you to consider how the out-of-school-time program is aligned with the regular school day.

School Program Alignment Question 1 Do leadership and staff of TX ACE and the regular school day leadership team have a shared understanding about student needs? Question 2 Is there a shared understanding of the direction that will be taken to address identified students’ needs between the regular school day and ACE program? Question 3 Do your activity selections reinforce students’ mastery of academic learning objectives (TEKS)?

School Program Alignment Question 4 Do ACE program personnel and the regular school day leadership and/or other school personnel engage in communication about the ACE program on a regular basis? Question 5 Do ACE program personnel and the regular school day personnel participate in shared learning experiences? Question 6 Is there shared ownership of problems and solutions (i.e. regular day, ACE and community)?

Recruiting and Retaining Program Participants The questions of this essential component of Implementation Practices are asking you to consider the data, processes, and strategies you use and develop to recruit and then retain students into the ACE program so that you are able to effectively meet your targeted number of students.

Recruiting and Retaining Program Participants Question 1 Do you use data regularly to guide decision-making about recruiting students into the ACE program? Question 2 Do you use data regularly to guide decision-making about retaining students in the ACE program? Question 3 Do you currently use processes, and strategies to recruit and retain students into the ACE program?

Recruiting and Retaining Program Participants Question 4 Do ACE program personnel, regular school day personnel, partners, and community members, each have a specific role related to recruiting and retaining students in the ACE program? Question 5 Are you currently serving the targeted population of students and families, what I am referring to are students most in need of academic assistance? Question 6 Do you aim to recruit and retain a diverse group of students into the ACE program?

Integrating Student & Family Voice and Choice The question of this essential component of Implementation Practices is asking you to consider how you currently integrate the opinions and desires of the students and their families about what they should or want to be doing in the ACE program.

Integrating Student & Family Voice and Choice Question 1 Do you integrate student and family voice and choice into the ACE program? Examples are: – A Student survey asking their opinions about various aspects of their experience in the program. – A Family survey and follow up report explaining how you used their feedback to improve the program – Student and family focus groups about how they feel and what they think about the ACE program. – Students have opportunities to express choice and share their voice in regard to what and how they are participating.

Ongoing Monitoring The questions of this essential component of Implementation Practices is asking you to consider how you monitor your program in on ongoing way throughout the entire academic year. What tools and resources do you currently use to monitor your program both at the “macro” (across all sites and overall site monitoring) and at the “micro” (individual program activities and staff) levels.

Ongoing Monitoring Question 1 Do you currently use data to regularly to drive decision-making about program and activity development? Question 2 Do you currently use observation protocols or checklists to assure fidelity of program and activity implementation? Question 3 Do you currently have a strategy to share program data regularly with students, families, regular school day personnel such as the school leadership and or principal, teachers, school board, other relevant, and community members?

Professional Development The questions of this essential component of Implementation Practices is asking you to consider how you incorporate and derive benefit from the ongoing professional development of all the people who work directly and indirectly with the students & families that you serve.

Professional Development Question 1 Do you use data regularly to guide decision-making about program staff skill development? Question 2 Do you currently have processes in place for responding to the skill development needs of program staff? Question 3 Is or was professional development and training effective in improving program staff skills?