1. NCES Winter Forum and 24th Annual Management Information Systems (MIS) Conference “Deep in the Heart of Data”
February 21-25, Austin, TX
ELLs, NCLB, and AMAOs: The WIDA Consortium’s approach to interpreting federal policy and providing guidance
Rahul Joshi, M.S.
Kristopher Stewart, M.A., M.P.A
WIDA Consortium, University of Wisconsin-Madison
H. Gary Cook, Ph.D., Research Director

2. No Child Left Behind
Under Title III, states must define annual measurable achievement objectives for LEP students served that relate to their development and attainment of English language proficiency while meeting challenging State academic content and achievement standards as required under Title I, section 3122 of NCLB.
Three specific AMAOs have been established under NCLB:
AMAO 1: ELL students progressing in English language acquisition
AMAO 2: ELL students exiting or reaching English language proficiency
AMAO 3: ELL Adequate Yearly Progress (AYP)
WIDA Consortium

3. Key Research
Linquanti, R. & George, C. (2007). Establishing and utilizing an NCLB Title III accountability system: California’s approach and findings to date.
Cook, H. G., Boals, T., Wilmes, C. & Santos, M. (2008). Issues in the development of annual measurable achievement objectives for WIDA consortium states.
Established Key Criteria for Setting AMAOs
WIDA Consortium

4. What is the WIDA Consortium?
24 states and D.C.
1.4 million students
WIDA Consortium

5. The WIDA ELP Standards
Standard 1 – Social & Instructional Language (SIL)
English language learners communicate for social and instructional purposes in the school setting.
Standard 2 – Language of Language Arts (LoLA)
English language learners communicate information, ideas and concepts necessary for academic success in the content area of Language Arts.
Standard 3 – Language of Mathematics (LoMA)
English language learners communicate information, ideas and concepts necessary for academic success in the content area of Math.
Standard 4 – Language of Science (LoSC)
English language learners communicate information, ideas and concepts necessary for academic success in the content area of Science.
Standard 5 – Language of Social Studies (LoSS)
English language learners communicate information, ideas and concepts necessary for academic success in the content area of Social Studies.
Within each standard, there are Model Performance Indicators (MPIs) for Listening, Speaking, Reading, and Writing for each grade-level cluster (PreK-K, 1-2, 3-5, 6-8, 9-12).
WIDA Consortium 5

6. Four Language Domains
Listening ─ process, understand, interpret, and evaluate spoken language in a variety of situations
Speaking ─ engage in oral communication in a variety of situations for a variety of purposes and audiences
Reading ─ process, interpret, and evaluate written language, symbols, and text with understanding and fluency
Writing ─ engage in written communication in a variety of forms for a variety of purposes and audiences
Listening/Speaking/Reading/Writing are the four main language skills. Listening and Reading are comprehension skills that involve receptive language; Speaking and Writing are communication skills that require productive or expressive language.
In order to give educators a comprehensive picture of ELL students’ English language proficiency – and to comply with federal law - all four domains are tested on the ACCESS for ELLs test and the W-APT screener.
For more information on the specifics of how each domain is tested, please refer to the other presentations in the toolkit related to the ACCESS for ELLs test and the W-APT screener.
WIDA Consortium 6

7. Organization of ELP Standards
Snapshot of a page of ELP Standards.
>four domains on the left side of the page
>Example Topics
>Proficiency Levels.
WIDA Consortium 7

8. WIDA Research Areas WIDA Consortium WIDA Research Team
Technical Assistance
Database Management
Applied Research
Basic Research
WIDA Consortium

9. Technical Assistance Projects and Policy Guidance – AMAO 1
Provide policy guidance on AMAO 1:
Determine the scoring metric
Determine the annual growth target
Set the starting point
Set the ending point
Determine the annual growth rate
Meet with the State stakeholders to discuss findings
State stakeholders make recommendations to SEA/LEA
WIDA Consortium

10. Technical Assistance Projects and Policy Guidance – AMAO 2
Provide policy guidance on AMAO 2:
Define ELP
Determine the cohort
Set the starting point
Set the ending point
Determine annual growth rate
Meet with the State stakeholders to discuss findings
State stakeholders make recommendations to SEA/LEA
WIDA Consortium

11. Sample AMAO 1 & 2 Analysis
This slide is a graphical example related to AMAOs showing growth which accompanies a technical assistance report to a Consortium member. This chart represents four important aspects: grade level cluster, percentile rank, proficiency level, and scale score gain. Cluster being examined includes the third, fourth, and fifth grade. A student’s starting composite proficiency level (<2.0 - <5.0) accompanied by the percentile rank on the x axis shows the ELL student’s growth. For example, when you subtract last year’s scale score from the present year, that will give a student growth, let’s say for a scale score gain of 32. Then, based on the fourth grade student’s composite proficiency level of 3.3, you can see that student’s is at the 60th percentile based on a scale score gain of 32 and starting proficiency level of 3.3 within the 3-5 grade cluster.
WIDA Consortium

12. Pits and Falls (and insights)
Lack of capacity at SEAs in data analysis
ELL policy:
Need for communication between entities
Influence federal ELL policy
Share outcomes, successes, results
Lots of data, but SEAs and LEAs know the kids best
WIDA Consortium

13. Data Requirements for AMAO Analyses
Longitudinal ACCESS test data  Students matched across years
Data reasonably cleaned and/or validated
Robust methodology for student matching
Reliable procedure to handle cases with missing/invalid student identification fields
On-demand, secure data delivery with remote availability
Provide both atomic and aggregate information
WIDA Consortium

14. Data Delivery - WIDA Data Warehouse
High-performance, scalable SQL Server database
Over a million individual test takers from 22 states across US (and growing…)
ACCESS Test Information (test scores, restricted student identification data and demographics),
Connected to selected NCES Research Data Collections
Core database for WIDA Projects, Research Initiatives and Reporting Framework
WIDA Consortium

15. WIDA Data Warehouse - Datasets
This slide indicates the main data components of the WIDA Data Warehouse.
The primary data source is the ACCESS for ELLs student performance data (indicated as 1) with restricted student identification information, for all the WIDA Consortium member states. Every student in this dataset has a unique student identifier and is longitudinally tracked across successive testing cycles.
The data warehouse also houses selected data elements from some of the publicly available research datasets collected by NCES namely: Common Core Datasets (indicated as 2), School and Staffing Survey Datasets (indicated as 3) and National Assessment of Educational Progress Datasets (indicated as 4). The Common Core Datasets are connected with the ACCESS datasets at both the school and district level, using unique State School ID and State District ID's. In this case, nationwide uniquely assigned School CCD Identifier is stored and also used for connecting these datasets.
Since School and Staffing Survey and National Assessment of Educational Progress Datasets are only available at the state level and hence connected accordingly. State level Common Core Data aggregates also form a part of the Common Core datasets.
WIDA Consortium

16. Building Longitudinal System for AMAO
It’s comprehensive and challenging, and why?
Across successive test administrations:
Old/Current Students Transfer to different state/ exit the ELL program
New students enter the state and/or the ELL program
Missing pieces in student identifier fields
Matching can be only correct up to a certain confidence level
Good quality data  Better understanding of student growth (and a key to happy life!)
WIDA Consortium

17. Building Longitudinal Student Record System
WIDA Consortium

18. Reporting Framework for WIDA Members
Statewide and WIDA-wide performance dashboard reports
More insightful reports soon based on ACCESS and NCES datasets
State/WIDA Dashboard Key areas
State Performance and State Growth in ACCESS Domains and Grade Clusters
Native Language Distribution for States
Comparison with NAEP Average Composite Scale Scores
Largest 10 Districts based on student enrollment
WIDA Consortium

19. State Performance by ACCESS Cluster
WIDA Consortium

20. State Growth by ACCESS Cluster
WIDA Consortium

21. Native Language Distribution in a State
WIDA Consortium

22. Pits and Falls (and insights)
Always question the data you have (for correctness and completeness)
Good quality source data High Reliability on student matching
Don’t assume quality of key student identifiers while longitudinally matching students
Make the framework inherently longitudinal
Data is a “double-edged sword:”
Good data MAY lead to good decision making and policies
Bad data CERTAINLY could lead to bad decision making on policies
WIDA Consortium

23. Questions? WIDA Consortium
Contact Information:
Rahul:
Kris:
Gary:
WIDA Consortium