1. Effectively Implementing Next-Generation Technology-Based English Language Learner (ELL) Assessments: Successes, Challenges, and Insights from Consortia States June 22, :00 – 2:30 P.M.

2. Summary Equitable educational opportunities for English language learners (ELLs) have long been a focus of State and Federal policy and legislation. Two English language proficiency (ELP) assessment consortia, the ELPA21 Consortium and the WIDA Consortium, were awarded grants from the U.S. Department of Education to design and deliver new technology-based ELP assessment systems. These systems are changing the way ELLs are assessed; introducing new technology, including online testing with advanced item types, tools, and accommodations, to benefit students and educators. Both consortia entered their first year of operational testing in 2015–2016, delivering new ELP assessments in a combined 48 member states across the nation. SEA representatives who have direct experience with the 2016 testing experience in their respective consortia, will share their successes, challenges, insights, and lessons learned in the first year of operational testing using the new technology-based ELP assessments.

3. Session Goals
Participants will learn about the successes, challenges, insights, and lessons learned in the first year of operational testing using the new ELPA21 & WIDA technology-based ELP assessments.
Participants will share their own experiences and gain new information and ideas from one another.

4. Participants
Colleen Anderson, Student Assessment Iowa Department of Education
Margaret Ho, ELPA21 Sustainability Director
Jobi Lawrence, EdD, Director, Title III, Education Program Consultant Iowa Department of Education
Carsten Wilmes, PhD, Director of Assessment, WIDA
Jared Wright, Test Administration Specialist, Assessment and Accountability, Utah Department of Education
Jenni Norlin-Weaver, EdD Data Recognition Corporation, Moderator
AND – All those present who shared our experiences this year

5. Agenda Introductions and Overview ELPA21 WIDA Successes Challenges
Insights
Lessons learned
“Harvesting the Wisdom of the Crowd”
Wrap-up

6. Similar? Different?
ELL Next Generation
Assessment

7. A New Assessment System for English Language Learners

8. Lead State Management Partner 3 Thought Partners Guidance and Feedback
Standards Authors
WHO? (con’t.) This is the Dream Team! The Oregon Department of Education is the lead state, and provides active and dedicated leadership in all areas of consortium work.
Management Partner (click) The Council of Chief State School Officers (CCSSO) is the same group that managed the development of the Common Core State Standards (CCSS).
Thought Partners (click) have provided expertise in each of their respective areas and have participated in various aspects of development.
The National Center for Research on Evaluation, Standards and Student Testing (CRESST at UCLA), is overseeing the validity studies and other psychometric work for the consortium.
NCEO at the University of Minnesota brings a wealth of experience working with underrepresented student populations to the consortium work.
As shown, an important ‘thought partner’ and resource for the ELPA 21 project is the Understanding Language Project at Stanford University. This partnership allows for the consortium to leverage the expertise and work Dr. Hakuta and colleagues bring to the focus on English language learners and learning.
As the Principal Investigator (PI) for ELPA21, Kenji Hakuta with Martha Castellon of the Understanding Language Initiative of Stanford University,
provides oversight on the development of the assessment system and
advises the consortium on best practices and emerging research around the development of English language proficiency.
Critical areas of PI input include
specifying the intended uses of ELPA21, 
providing guidance in the development of assessments that align with the ELP standards and descriptors of performance level competencies. 
UL’s focus on language, literacy, and learning in the content areas is done through:
Papers: Practice in the content Areas, language and literacy, policy and building system capacity
Teaching Resources: Newsletters, Massive Open Online Courses (MOOCs), Teaching resources that exemplify high-quality instruction for ELLs across English Language Arts, Mathematics, and Science
Dissemination: Consistent coverage in Education Week, AERA journals, and professional education publications
Policy Development: Statements clarifying important issues related to the new standards and the role of language, such as the work with CCSSO
Author (click) While WestEd was contracted by the ELPA21 consortium to author the new ELP standards, practitioners from states have played a very active role in reviewing and providing feedback as various drafts were made available. (click)
Guidance and Feedback

9. ELPA21 Consortium Overview
In 2012, an Enhanced Assessment Grant was awarded to the Oregon Department of Education in partnership with CCSSO, CRESST, NCEO, West Ed, and Understanding Language. Originally, 11 states partnered in the EAG grant.
ELPA21 is a consortium of diverse states, diverse in terms of geography, membership in other consortia including independent states, states that have and haven’t adopted the common core, etc.
ELPA21’s Mission commits to: Acknowledging the diverse and rich language experiences English language learners (ELLs) bring to school, we recognize their English language proficiency is constantly growing. ELPA21 measures that growth based on the new English Language Proficiency Standards and provides valuable information that informs instruction and facilitates academic English proficiency so that all ELLs leave high school prepared for college and career success.
On May 7, 2015 ETS and ELPA21 were awarded a silver medal for item development work that is fully compliant with the Accessible Portable Item Protocol (APIP 1.0) interoperability standard.ELPA21 partnered with ETS for item development and states are working with a variety of vendors for test administration and scoring
Item writers and reviewers are professional educators of ELs in Consortium partner states, and are highly trained by ELPA21
The work was a collaboration effort with Task Management Teams/ Task Forces, Executive Board, and the Consortium Council

11. WIDA Consortium Overview
In 2002, an Enhanced Assessment Grant was awarded to the Wisconsin Department of Public Instruction, WIDA's first home. Three states were involved in the grant: Wisconsin (WI), Delaware (D), and Arkansas (A), so the acronym WIDA was chosen for the name. Later, World-class Instructional Design and Assessment was created to fit the acronym. More recently WIDA decided to stop using the acronym definition.
WIDA’s Mission describes their work:
WIDA advances academic language development and academic achievement for linguistically diverse students through high quality standards, assessments, research, and professional development for educators.
WIDA spans 38 states
ACCESS for ELLs is a standards-based, criterion referenced English language proficiency test designed to measure English language learners' social and academic proficiency in English. It assesses social and instructional English used within the school context as well as the language associated with language arts, mathematics, science, and social studies across the four language domains (Listening, Speaking, Reading, Writing).
ACCESS is just one of many products/services available to members
WIDA partners with the Center for Applied Linguistics (CAL) for test development and with Data Recognition Corporation (DRC) for ACCESS 2.0 test administration and scoring
Item writers and reviewers are professional educators of ELLs in Consortium partner states

12. ELPA21 Successes Increased Access Braille
Students With Significant Cognitive Disabilities
Accommodations & Accessibility Features
Increased Student Engagement through Technology Enhanced Items and Computer Based Delivery System
First Operational Assessment in Spring of 2016
Cohesive system to integrate with existing district policies and enhance the student’s experience through strong engagement, and minimal testing time
High-quality communications for the states’ adaptation for constituents and high quality resources for professional development around both the standards and the assessment.
Sustainability to respond to states’ need for valid and reliable assessment, maximize information, and minimize cost
Six grade bands (K, 1, 2–3, 4–5, 6–8, and 9–12)
2 fixed forms per grade band

13. WIDA Successes
Schools across the country began using WIDA’s new online assessment option, ACCESS for ELLs 2.0
4,143,729 online test sessions completed
Increased student engagement through a more dynamic testing experience
Built-in accommodations and accessibility features appropriate for a range of student needs
Staged adaptivity captures students’ individual range of language skills

14. ELPA21 Challenges State Policies and Practices
Ongoing calibration amongst states in a changing political climate and leadership transition in partner states
State adoption processes and timelines
MOU alignment with all partner state policies and requirements
Paper & pencil issues
Attemptedness policies
Timelines
Double Testing ELs
Field test numbers were low due to the requirement to double test ELs, due to high stakes decisions related to the Summative and Screener assessments for ELs.
Supporting Change in the Field (e/g/ moving from paper & pencil to online testing)

15. WIDA Challenges Policy/Practice Differences
Ex: Attemptedness, Timed Test, Data Validation, Dual Paper and Online Administration
Issues Around
Communication
Technology and Test Preparation
Customer Service

16. ELPA21 Insights Stakeholder Engagement is Key to Sustainability
Practice Tests (especially for K-2 students) are critical to mastering the technology needed to engage in a computer based assessment
Alignment of Processes and Business Rules Across Partner States and Across Vendor Platforms Critical
Over-Communication to the Field is Necessary
Certification training
Plan for the “bumps in the road”

17. WIDA Insights Communication, Communication, Communication
Documentation of Each State’s Assumptions and Differences
SEA Listening Sessions
Webinars
Quarterly and Regional Calls
Individual State Listening Sessions
LEA Survey

18. WIDA ACCESS2.0 Student Insights
Student experience
“Students liked the online test, some indicating that was the case even if there were technology difficulties.”
“Students were able to feel more confident about the test.”
“Pictures and people talking to them made it more interactive, which resulted in students being more motivated and interested.”

19. ELPA21 Lessons Learned
Survey Results of First Operational Summative Assessment Just Released
Communication is Key
Training and Follow Up Needed
Stay Tuned…
Screener Release Scheduled for Spring of 2017

20. WIDA Lessons Learned Check Assumptions
Ensure Site Technology Readiness
Proactive planning
Improved test management and testing experiences
Include SEAs / LEAs in feature specification and testing
Hands-on preview
FREQUENT communication

21. Media Rich Content and Responses
Typical State Assessments
WIDA – ACCESS for ELLs 2.0
Type of
Test
Average Form Size per Test
Average Size of Each Response
Fixed Form Test
2 MB
Non-Writing - 2 KB
Writing - 25 KB
Writing (Paper Response)
10 MB
5 KB
Writing (Online Response)
25 KB
Speaking
28 MB
6 MB (to TSM) 600 KB (from TSM)
Computer Adaptive Test
700 KB
Response – 5 KB Item – 25 KB
Listening
Response – 5 KB Item – 2 MB
Reading
IMPORTANT NOTE: The higher the complexity of the test content and the response formats, the more bandwidth that is needed at the testing sites.

22. Keys to Success
Technology
Test Preparation
Customer Service

23. Similar? Different?
ELL Next Generation
Assessment

24. YOUR Experiences?

25. Next Generation ELL Assessment
Takes advantage of available technology
Richer assessment
Speaking responses
Benefits students, educators and families
Better information
Policy and Practice Synergies
Focus on ELLs and their needs