1. Child Outcomes State of the State
DEC Conference
10/15/2010
Child Outcomes State of the State
Council for Exceptional Children/Division of Early Childhood
Conference
October 2010
Kim Carlson, Asst. Director/619 Coordinator
Ohio Department of Education
Office of Early Learning and School Readiness
Ohio Department of Education

2. Ohio: Demographics
6th Largest State for children with disabilities (ages 3-21)
12/09 child count: 23,246
612 School Districts
88 Counties
Education Service Centers
County Boards of DD
16 State Support Teams-regionalized TA/PD

3. Ohio Assessment Policies
State statute requires programs to document child progress using research-based indicators prescribed by the Department and report results annually.

4. Accountability & Continuous Improvement
Ohio Early Learning Program Guidelines:
Systematically assess conditions, practices, policies and program performance
Conduct annual self-assessment with identified stakeholders and community partners to evaluate accomplishment of program goals
Data reported to ODE
Child progress data
Program licensure report
Family input and feedback
Complaint records
Fiscal reports
Transition services

5. Early Childhood Outcomes Summary Form
Initial year of the COSF: 2007
Trained local teams of teachers and administrators
Locals report scores to state
Fall & spring of every year
State does conversion of COSF scores to progress categories
Implementing alternate form spring 2011
Source summary OR
Evidence statements

6. Early Childhood Outcomes Summary Form
Multiple Sources Required
Two Sources are consistent statewide for funded programs
Get It! Got It! Go! &
Ages/Stages Questionnaire: Social-Emotional (completed with parent)

7. Measurement System Knowledge & Skills
Preschool special education supervisors
Annual series of communication
Fall Data/Reporting Requirements
February Supervisor Discussions
Spring Institutes
Two Institutes
Stage 1: Data Quality and Accuracy
Stage 2: Data Analysis and Public Reporting
For families: outcomes brochure/cards

8. Measurement System Knowledge & Skills
Data collection protocols
Data checks in the state data system
Funding impact

9. Data Collection: Efficient, Effective, Accurate

10. Measurement System Knowledge & Skills
New Web page
Webinar series (in development with ECO Center)
Teacher reference sheet
Approved PD through regional state support teams
Alternate COSF form- 2 options

11. Teacher Reference: PROCESS Implementation

12. ECO Implementation Survey
Purpose: Identify challenging practices for local districts and personnel
Survey May 2010
Lessons Learned:
Multiple sources of information
Duplication of source summaries
Comparable to same-aged peers
Use of Decision Tree
Fidelity of Process
Data Issues

13. Use of the Decision Tree
2010 ECO Conference
How Will I Know?

14. ECO Implementation Survey
Lessons Learned
Core team input
Preschool special education teacher
General education teacher
Parent
Related services personnel
Involving parent

15. Reasons for Completion without all Members Contributing
2010 ECO Conference
How Will I Know?

16. Involving Parents
2010 ECO Conference
How Will I Know?

17. WHOLE
Whole child
Whole class
Whole system
How is this in evidence?

18. Accountability Framework Early Childhood
DEC Conference
10/15/2010
Accountability Framework Early Childhood
Child and Family Outcome Measures
Literacy: GGG and KRA_L
Early Childhood Outcomes Summary
Parent Survey
Health & Developmental Screenings
Aligned with P-12
Content Standards
Our Early Learning Programs fall within an accountability framework – measuring outcomes at the child and family levels; the classroom level and at the program level.
Program capacity measures: ELLCO, Self-reflection planning tool, Step Up,
Curriculum embedded performance measures: ELI programs asked to identify a curriculum and measures for it. They are to align their curriculum with Ohio’s early learning content standards and encompass all developmental domains. They most often use creative curriculum or high scope. We are working to help identify more curriculum embedded performance measures.
Child/Family Outcome measures include: GGG, KRAL—will include health and developmental screenings, parent survey
Most commonly used curriculums for preschool = Creative Curriculum & High Scope
Most commonly used PSE = Creative & AEPS
Program Capacity Measures
Curriculum-Embedded Performance Measures
ELLCO: Early Language & Literacy Classroom Observation
ELLRT: Early Language & Literacy Reflection Tool
Self-Assessment Tools
Tools for Alignment to the Early Learning Content Standards
IEP Outcomes
Ohio Department of Education 18

19. State Infrastructure Regionally Based/Local Needs
Professional Development
Preschool Literacy Core Curriculum (CORE)
SIRI: Pre K/K State Institutes for Reading Instruction
Teacher Leaders
Ohio Early Learning Content Standards
Integrated Curriculum
The IEP and the Early Childhood Curriculum
Assessment Training
Observation and Mentoring
ELLCO Observations and TA
English Language Learners
Regional Leadership Meetings

20. PD Connection IVDL/Higher Education Faculty
At no cost unless college credit desired
Institute of Education Science Grant- APPLE: Ohio
Evaluate statewide professional development in Ohio
80 Teacher Leaders each year
Group A will participate in the Preschool Literacy Core
Group B will participate in Preschool Literacy Core with Teacher Leader support
Group C will participate in two of the Ohio Early Learning Content Standards modules (Math, Science, Social Studies)

21. Ohio Resource Center for Mathematics, Science ad Reading
Resources for early childhood
Early Childhood Bookshelf
Standards-aligned and Peer Reviewed Lessons
Early Childhood Building Blocks- Best Practice Articles

22. DEC Conference
10/15/2010
What do we know?
Children served in Public Preschool and PSE classrooms experienced statistically significant growth from fall to spring
Public Preschool participants had significantly higher KRA-L scores than all other K students.
PSE participants who are nondisabled in K have higher KRA-L scores than all other K students who are nondisabled.
We looked at the Fall 2007 KRAL results (FY08) for children participating in state funded preschool programs.
KRA-L Notes:
1. Probability levels: The higher KRA-L scores for the differences expressed above comparing ECE or PS-SE students to other kindergarten students are all statistically significant at p < .01. (T-test and ANOVA analyses).
2. KRA-L Statistics:
Former ECE students (N=3266, Mean = 21.03, SD= 6.02)
Former PS-SE students not classified as SWD (N=4071, Mean = 22.02, SD= 5.88)
All kindergarten students from neither ECE nor PS-SE programs (N= 113,035, Mean =19.77, SD = 6.94)
All kindergarten students from neither ECE nor PS-SE programs and not classified as SWD (N= 108,039, Mean = 19.96, SD= 6.85.
Ohio Department of Education

23. DEC Conference
10/15/2010
Early Language and Literacy Classroom Observation (ELLCO) Study Results
Instrument
Methodology
Classrooms rated on 14 individual ELLCO elements
5 point evidence scale (5 = exemplary or strong evidence)
2 subscales:
General Classroom Environment
Language, Literacy and Curriculum
Stratified random sample of PSE classrooms
2009: 115 observations in PSE
2010: 299 observations conducted
The ELLCO is study conducted by Learning Point Associates on behalf of ODE. We have had a focus on PSE classrooms.
The tool is a published (Brookes) observation tool that focuses on the quality of the literacy classroom environment.
Research questions focused on:
What is the quality of literacy practices in ODE funded programs?
How do the scores of programs and/or classrooms compare to recognized standards of effective literacy practice, as defined by a 4 or 5 on the ELLCO rating scale?
Based on ELLCO results, what are statewide or local PD needs?
2 well-trained observers were sent into each selected classroom to watch minutes of the classrooms, followed by a 10 minute interview. Observers watch same activity simultaneously and must reach consensus on the individual classrooms ratings following the observation.
Ohio Department of Education

24. ELLCO Study Questions
What is the quality of literacy practices in ODE-funded early education programs?
How do the scores of programs and classrooms compare with recognized standards of effective literacy practice (defined as a 4 or 5 on the ELLCO rating scales)?
To what extent are teachers (best practices) and programs (capacity) making progress toward reaching or exceeding recognized standards of literacy practice that promote literacy success?
Based on the ELLCO results, what are statewide or local professional development needs?

25. ELLCO Mean Ratings for PSE Classrooms, Spring 2010, N=299
DEC Conference
10/15/2010
ELLCO Mean Ratings for PSE Classrooms, Spring 2010, N=299
Learning Point Associates examined mean ratings and sum scores of PSE classrooms to determine the quality of literacy practice at the state level. As shown in Figure 3, average ratings on a five-point scale in Spring 2010 were 3.94 for the General Classroom Environment Subtotal; 3.63 for the Language, Literacy, and Curriculum Subtotal; and 3.71 for the Classroom Observation Total.
Ohio’s Early Learning Program Guidelines established an average rating of 4 as the goal for all PSE teachers. On average, the quality of the classroom environment and literacy practices was rated at the “Basic,” (3) level, although the General Classroom Environment score was only slightly below the ODE-desired standard of 4 or higher.
Ohio Department of Education

26. ELLCO Mean Ratings for PSE Classrooms, Spring 2009
DEC Conference
10/15/2010
ELLCO Mean Ratings for PSE Classrooms, Spring 2009
STANDARD EFFECTIVE PRACTICE
The observations conducted in the spring of 2009 demonstrated average ratings on a five-point scale of 4.11 for the General Classroom Environment Subtotal; 3.74 for the Language, Literacy, and Curriculum Subtotal; and 3.82 for the Classroom Observation Total.
N=115
*N=116
Ohio Department of Education

27. ELLCO Results 2010 Highest-rated ELLCO elements:
DEC Conference
10/15/2010
ELLCO Results 2010
Highest-rated ELLCO elements:
Classroom Management Strategies (Mean=4.29)
Classroom Climate (Mean=4.24)
Organization of the Classroom (Mean=3.99)
Lowest-rated ELLCO elements:
Presence and Use of Technology (Mean=3.18)
Recognizing Diversity in the Classroom (Mean=2.84)
In general, the highest rated ELLCO elements and lowest rated elements are the same as the previous year.
Ohio Department of Education

28. Summary of ELLCO Results 2010
DEC Conference
10/15/2010
Summary of ELLCO Results 2010
41% PSE classrooms demonstrated effective literacy practices
Higher levels of teacher educational attainment were associated with higher ELLCO ratings for all three scores
Average early childhood education teaching experience was approximately 12 years
Teachers with 13 or more years of teaching experience scored significantly higher on the Language, Literacy, and Curriculum Subtotal & Classroom Observation Total than teachers with 6-12 years of teaching experience
Higher ELLCO ratings are associated with classrooms that have a smaller percentage of students with disabilities
ELLCO findings suggest that within preschool special education classrooms across the state, the quality of literacy instruction and classroom environments is slightly below the designated level of proficiency. Overall, 41 percent of PSE classrooms demonstrated effective literacy practices, and 83 percent were either at the “Basic” or “Proficient” levels.
The Classroom Observation Total mean rating was 3.71 in Spring This rating was slightly lower than 4.00 (the rating designated by ODE as demonstrating effective literacy practices).
In addition, the quality of literacy instruction stayed consistent from Spring 2009 to Spring 2010.
The mean rating from Spring 2010 was slightly lower than the mean rating from Spring 2009; though not statistically different.
As in Spring 2009, PSE classrooms continued to score higher on the General Classroom Environment Subtotal than on the Language, Literacy, and Curriculum Subtotal.
Classrooms with teachers with master’s degrees received significantly higher ratings than classrooms with teachers with bachelor’s degrees.
Ohio Department of Education