Classroom Self-Assessment Using the ICP: Evaluation of the Process Award of Excellence in Inclusion of Children with Special Needs ExceleRate Illinois

State Context – ExceleRate Illinois Governor’s Office of Early Childhood Development (OECD) All state agencies providing services to young children and their families Shared standards – ExceleRate Illinois (QRIS – Quality Rating & Improvement System) Public schools Head Start Child care

One of Five Awards of Excellence

The Award of Excellence in Inclusion … GOES BEYOND compliance to focus GOES BEYOND compliance to focus on quality of inclusive practices on quality of inclusive practices Based on SELF-STUDY AND Based on SELF-STUDY AND PORTFOLIO DEVELOPMENT PORTFOLIO DEVELOPMENT Considers inclusive practices in Considers inclusive practices in both PROGRAMS and CLASSROOMS both PROGRAMS and CLASSROOMS

Application Process Self-Study and Portfolio Development, with Supports

Framework for Demonstrating Excellence in Inclusive Practices Five Program Standards Program Practices (IIGC) Classroom Practices (ICP) AoE Supports Self-Study Portfolio Self-Assessment & Goal-Setting Self-Assessment & Goal-Setting

Standards, guidelines & practices: self-assessment at 3 levels Five Standards Program level Commitment to including children with special needs (Standard 1) Staff training related to applying for this Award (Standards 3, 4, 5) Standard 2 – accomplished through next level (IIGC) 21 Item IIGC Program level (Standard 2) Criteria for high-quality inclusive practices program-wide Item 21 – accomplished through next level (ICP) 12 Item ICP Classroom level – addresses Item 21 of IIGC Criteria for high-quality inclusive practices in individual classrooms

FROM CLASSROOM ICP TO PROGRAM SELF-STUDY PORTFOLIO ICP SCORE SHEET (all indicators marked) PRIORITIES: Worksheet 5 CLASSROOM GOALS & PLAN: Worksheet 6 IIGC ITEM 21 PROGRAM PORTFOLIO

Completing an APPLICATION for an AoE in Inclusion Review of Portfolio by State Advisory Panel Review of Portfolio by State Advisory Panel Complete self-assessment on Standard 1 (Commitment to Inclusion – prerequisite for applying for the Award) Complete self-assessment on Standard 1 (Commitment to Inclusion – prerequisite for applying for the Award) Complete self-assessment on Standards 3, 4, 5 (Training in Inclusion) Complete self-assessment on Standards 3, 4, 5 (Training in Inclusion) Join Cohort, Complete Award of Excellence Training Join Cohort, Complete Award of Excellence Training Initiate Self-Study Portfolio using Illinois Inclusion Guidelines (IIGC) & Inclusive Classroom Profile (ICP) Initiate Self-Study Portfolio using Illinois Inclusion Guidelines (IIGC) & Inclusive Classroom Profile (ICP) Connect with AoE Staff Continue/Complete Program Self-Study Portfolio Continue/Complete Program Self-Study Portfolio Submit Portfolio to OECD Submit Portfolio to OECD

Preparing for Self- Assessment with the ICP Administrators in applicant programs On-line introduction from FPG Full-day training, including ½ day on the ICP and supporting staff in using it for self-assessment Follow-up support from AoE staff Classroom staff 1-hour webinar on the ICP and using it for self- assessment

Evaluating Use of the ICP for Self-Assessment How well did it work?

Evaluation Procedures External Evaluator Data Sources Surveys (1-5 ratings: administrators, classroom staff) Interviews (open-ended questions: administrators, classroom staff) Observations using the ICP (external observers)

Evaluation Questions – Focus on the ICP Preparation for ICP self-study Did classrooms and programs feel prepared? Completing the ICP self-study How did classrooms approach the task of self- assessment? How well did it work? Validity of the ICP self-study process Did the ICP self-study provide an accurate picture of high-quality inclusive practices?

Participants in Post-Award Evaluation (thus far) External evaluation completed on Cohort 1 8 programs (7 public schools; 1 combination Head Start/child care from pilot of process) ICP data from 14 classrooms (classrooms in pilot program did not participate in evaluation)

Q1 – What helped prepare us to use the ICP for self-assessment? SURVEY – Scale of 1 [disagree] – 5 [strongly agree] Administrators (% with 4/5 rating) half - day training on ICP by Award of Excellence staff “prepared me to support classrooms on ICP” - 67% Classroom staff (% with 4/5 rating) 1-hour webinar with Award staff – 54% “prepared us to use the ICP for self-assessment” – 54% ICP Manual – usefulness for completing self-assessment examples provided under rubrics – 93% clarification pages – 87%

Q2 – What process was used to complete the ICP in classrooms? How did administrators support the process? Facilitated process with timeline, substitute teachers, team time (most) Facilitated group discussion of webinar (some) Participated in classroom team discussions (some) Observed with ICP, shared observations with team (one)

Q2 (continued) How did classrooms approach the task? As a classroom team (teacher [100%], assistant teacher [93%], related service staff [86%], supervisors [43%]) Variety of data sources (80% used all of these - observation, document review, discussion, lead teachers’ knowledge of classroom) Primary data source (discussion: 54%; observation: 23%) Time-frame considered (92% considered inclusive practices implemented during whole year) Children considered (69% considered all children with special needs in classroom)

Q3 - How well did the results represent practices? Administrator perspective (4/5, scale of 1 [disagree] – 5 [agree]) Accurate picture of classrooms – 86% Meaningful goals established from use – 100% Classroom staff perspective Accurate picture of classrooms – 87% Meaningful goals established from use – 93% Important contribution to overall Portfolio – 93%

Q4 – Validity from Perspective of External Observers Average ratings across classrooms and items Classroom self-ratings = 6.3 (range 4-7) Observer ratings = 5.0 (range 3-6) Scores on individual items Classroom self-ratings – all but two items items (activity adaptations, transitions) averaged 6 or above Observer ratings – only 3 items [space/materials adaptations; conflict resolution; monitoring] averaged 6 or above Overall correlation between self-assessments and observer ratings – no significant relationship

Items with closest agreement and most difference Closest agreement Conflict resolution Adaptation of group activities Most difference Family-professional partnerships Adult involvement in peer interactions

Is There Value in Self-Assessment Using the ICP? YES Classrooms and programs found value in using the ICP for self- assessment, and felt they learned from the process Raised awareness of evidence-based practices Promoted good discussion as a team Promoted changes in what they were doing BUT Overestimated own inclusive practices Results indicate less than full understanding of high-quality inclusive practices, even in generally high-quality programs Additional training and other supports needed to obtain full benefit from the process

Looking Forward Continue to use the ICP for raising awareness of inclusive practices and for self-assessment Improve quality and value of self-assessment Pursue deeper, more accurate understanding of the ICP items (training, coaching) Provide better guidance for program administrators in valuing and supporting the team process

For More Information … AoE in Inclusion of Children with Special Needs Website: aoeinclusion.com ExceleRate Illinois –