1. Update on Development of the Smarter Balanced Common Assessment February 14, 2012

2. 2 Welcome, the webinar will begin at 2:30 Pacific. Please mute your audio so that your personal conversation is not heard by others. Please use “Chat” to ask that a question be addressed.

3. 3 Introduce Oregon staff engaged with SBAC Welcome participants and guest, Tony Alpert Quick Background (Kathleen Vanderwall) Recent Questions impacted by state policy (Derek Brown) Timeline, Evidence Centered Design, Process for Answering Questions (Tony Alpert) Open Questions

4. 4 Accessibility and Accommodations 1 Formative Assessment Practices and Professional Learning 2 Item Development 3 Performance Tasks 4 Reporting 5 Technology Approach 6 Test Administration 7 Test Design 8 Transition to Common Core State Standards 9 Validation and Psychometrics 10 Oregon Workgroup Members Advising SMARTER Balanced

5. 5 Dianna Carrizzales-Engelmann Accommodations and Accessibility Co-Chair Mark Freed Reporting Co-Chair Steve Slater Validation and Psychometrics Co-Chair Kathleen Vanderwall Item Development Member Rachel Aazzerah Test Administration Member Jim Leigh Transition to the Common Core Member

6. 6 Participants Tony Alpert Chief Operating Officer SMARTER Balanced Assessment Consortium (SBAC)

7. Quick Background Kathleen Vanderwall Manager, Test Design

8. Oregon Adopted CCSS Superintendent of Public Instruction Susan Castillo “By joining other states in the Common Core movement, we better position our students to compete nationally and internationally with their peers for jobs in the 21st century workplace.” Oregon Board of Education adopted Mathematics and ELA & Literacy CCSS in October 2010 Operational Assessment begins 2014-2015 www.ode.state.or.us/go/commoncore

9. Oregon is a governing member of SBAC The SMARTER Balanced Assessment Consortium (SBAC) is one of two multistate consortia awarded funding from the U.S. Department of Education to develop an assessment system based on the new Common Core State Standards (CCSS). One Aspect of the Summative Assessment Mandatory comprehensive accountability measures that include computer adaptive assessments and performance tasks, administered in the last 12 weeks of the school year in grades 3–8 and high school for English Language Arts (ELA) and mathematics http://www.k12.wa.us/SMARTER/

10. CCSS Implementation Stewardship Team CCSS implementation efforts require a group of committed individuals that represent all sectors of education in order to achieve the overarching Common Core goal: every Oregon student College and Career-ready. Called the Stewardship Team, this group consists of 39 stakeholders, identified through a statewide nomination process, and appointed by the Superintendent of Public Instruction. Membership includes K-12 and postsecondary educators in English language arts, mathematics, science, social science, and career and technical education Early childhood educators, ESD staff Faculty from colleges of education (teacher preparation) Special education and English Language Learner directors Representatives from business, Oregon PTA, and professional educational organizations

11. Recent Questions Impacted by State Policy Derek Brown Manager, Assessment of Essential Skills

12. 12 Does the question pertain to assessment? SBAC is in the questioning period itself, procuring contracts for test design, communication etc. Does the policy question pertain to Oregon specific requirement (such as the Oregon Diploma)? ODE’s process includes: ODE Staff Recommendation with supporting data, Review by appropriate stakeholder group (i.e. AESRP), Recommendation to State Board of Education

13. 13 Q: Will SBAC replace OAKS? A:Yes, we believe so. ODE will institute the SBAC assessment to replace OAKS in reading, writing, and math starting 2014-15. Q: Does SBAC assessment become the ticket to HS diploma? A: Once ODE staff have the fundamental information needed about this assessment then AESRP will evaluate and recommend an appropriate achievement level to SBE.

14. 14 What about 11 th graders who take OAKS in 2013-2014 and don’t pass….will they or won’t they have opportunity on SBAC as seniors towards their diploma? We do not have an answer at this time, we don’t yet know about SBAC eligibility. Anticipate that ODE will give equitable opportunity.

15. 15 What about 9 th or 10 th graders who already passed OAKS in high school prior to taking SBAC as 11 th graders in 2014-15? No firm policy yet, but we will bring this up with stakeholder groups like AESRP and Accountability Advisory Will their OAKS tests count towards diploma? No policy yet, ODE will engage stakeholders. We anticipate feedback that would encourage ODE to write such policy.

16. Development Timeline Tony Alpert Chief Operating Officer, SBAC www.smarterbalanced.org

17. 17

18. 18 GradesSummativeInterim (Optional) Formative Tools and Professional Learning (Optional) ✔✔✔ 1-2 Performance Tasks as Required to Cover CCSS ✔ EOC and Comprehensive ✔ ✔✔ EOC and Comprehensive ✔ Optional ✔ EOC and Comprehensive ✔ 3 8 9 10 11 12

19. 19 SBAC is committed to using evidence-based design in its development of the common assessment: Claims are the broad statements of the assessment system’s learning outcomes Each claim requires evidence that will support interpretations of competence towards achievement of all claims Relevant and sufficient evidence needs to be collected in order to support each claim

20. Evidence-Centered Design (from SBAC Item Spec showcase #1) 20

21. How to view Evidence-Centered Design (ECD)?  ECD is as a logical, systematic approach to test design and development based on the principles of evidentiary reasoning.  ECD is not used as a rigid set of procedures.  ECD gives us tools for linking all of the steps of test development to the collection of the evidence required to support the claims to be made about test takers. 21 Evidence-Centered Design

22. What does ECD do?  ECD does not make the traditional test development steps obsolete.  ECD offers tools to help get the steps done.  ECD helps clarify, document, and link the steps in test development, but the steps still need to be done whether ECD is used or not. 22 Evidence-Centered Design

23. Steps in ECD ECD starts with claims about the KSAs of test takers—what SBAC wants to say about students on the basis of assessment results. SBAC has provided high-level claims for summative tests in ELA and Mathematics. ECD requires that evidence be provided to support each claim. SBAC has provided descriptions of the evidence needed to support the claims. The SBAC Content Specifications provide the claims and descriptions of evidence needed as the basis of ECD. 23 Evidence-Centered Design

24. Steps in ECD  Each item/task (SR, CR, PT, TE) has to elicit evidence for one or more claim(s).  Item Specifications will set specifications for the tasks, and rules for their accessibility and fairness. Each task will be linked to evidence for a claim or claims. 24 Evidence-Centered Design

25. Interconnectedness of ECD Steps  Earlier decisions affect all later phases.  Any later phase could require changes in earlier phases.  Phases provide information for each other, both forward and backward.  Iteration and recycling throughout the phases is expected and helpful. 25 Evidence-Centered Design

26. Benefits of ECD  ECD strengthens validity argument.  ECD aligns all parts of the test design and development process.  Every step of test development from setting specifications to reporting scores is logically linked to previous and following steps. 26 Evidence-Centered Design

27. 27 Theory Of Action and CCSS Content Specifications Item and Test Specifications Rapid Prototyping Initial Item Writing Alignment and Bias Review Item Piloting Large Scale Item Writing Large Scale Alignment and Bias Review Field Testing and Scaling

28. 28... the SMARTER Balanced Assessment Consortium launches a new website: www.smarterbalanced.org Visitors will be able to: Download resources and new fact sheets Sign up for a monthly e-newsletter Explore an interactive timeline of activities by school year Provides frequent updates on activities, milestones, events Showcases the innovative work of the Consortium

29. 29 Submit questions to holly.carter@state.or.us Answering through review of SBAC’s RTTT Proposal to US DOE Contract Technical Advisory Committee Governing State Votes and confirmation through SBAC.

30. Derek Brown, Manager of Assessment of Essential Skills Kathleen Vanderwall, Manager of Test Design