2011 SMARTER Balanced Assessment Consortium: Technology Update November 21, 2011

Announcements This webcast is being recorded. The audio visual file and presentation slides will be available on the SIIA Education Division home page. Submit questions at any time via the “Chat” feature. SIIA Events: – SIIA Ed Tech Business Forum ONLINE, Nov 28–29, 2011 – SIIA Member Webcasts Dec. 6 - Opportunities for Entering the US K12 Market Dec. 6 – CCSSO Blue Box: Common Ed Data Standards and Impact on the Application Marketplace Dec. 7 – Ed Tech Opportunities in Africa – BETT 2012 in London Sign Up for SIIA Working Groups and Committees 2

Agenda 1.Welcome & Introductions 2.Overview of SBAC organizational structure 3.Overview of SBAC philosophy 4.Key technology issues around adaptive testing, item types, school implementation 5.Detail around architecture and interoperability 6.Q/A 3

Speakers Tony Alpert, COO, SBAC Jill Abbott, Executive Director, SBAC System Architecture Development, Measured Progress Moderator: Mark Schneiderman, SIIA 4

Measured Progress ©2011 SMARTER Balanced Assessment Consortium IT Systems Architecture Conversation with SIIA November 21, 2011

2011 Agenda Overview of SBAC organizational structure Overview of SBAC Key technology issues Detail around architecture and interoperability Q/A

Measured Progress ©2011 SBAC Organizational Structure

2011 SBAC Organizational Chart

2011 Accessibility and Accommodations Formative Assessment Practices and Professional Learning Item Development Performance Tasks Reporting Validation and Psychometrics Technology Approach Test Administration Test Design Transition to Common Core State Standards SBAC Work Groups

Measured Progress ©2011 SBAC Overview

2011 Collection of 29 member states Collaboratively working to develop a student assessment system aligned to a common core of academic content standards Innovation The goal of the Consortium is to ensure that all students leave high school prepared for post-secondary success in college SBAC Overview

2011 To develop a set of comprehensive and innovative assessments for grades 3-8 and high school in English language arts and mathematics aligned to the Common Core State Standards Students leave high school prepared for postsecondary success in college or a career through increased student learning and improved teaching The assessments shall be operational across Consortium states in the school year SBAC Purposes

An integrated system 2.Evidence of student performance 3.Teacher involvement 4.State-led with transparent governance 5.Continuously improve teaching and learning 6.Useful information on multiple measures 7.Adheres to established professional standards Seven Key Principles

2011 Balanced Assessment System Common Core State Standards specify K-12 expectations for college and career readiness Common Core State Standards specify K-12 expectations for college and career readiness All students leave high school college and career ready Teachers and schools have information and tools to improve teaching and learning Interim assessments are flexible, open, and provide actionable feedback Summative assessments are benchmarked to college and career readiness Teachers can access formative assessment practices to improve instruction

Measured Progress ©2011 Key Technology Issues

2011 Timeline and budget Interoperability Planning now for the future Compatibility Security Overall

2011 Maturity in the marketplace Lack of interoperability Types of scoring – Machine vs. human – Performance tasks Comparability Adaptive Testing and Scoring

2011 Item interoperability Defining innovative item types Technology enhanced items Enabling and allowing for multiple representations Item Types

2011 Capacity – Hardware – Software – Bandwidth – Technology support Policies – Testing window – Implementation – Scoring, reporting and data – Uses: More than just assessment School Implementation

Measured Progress ©2011 Architecture and Interoperability

2011 The Approach: This is larger than any single company agenda; it requires an open and collaborative approach The Team: We have assembled a Collaborative with: The thought leadership (industry, academia, industry associations & individual contributors) to ensure the vision is grounded in assessment and education best practices The technical capability to deliver The Process: Experienced and Agile, sustainability via a community approach Overview of SBAC Architecture Initiative

2011 Assessment Lifecycle Content Development Content Development Pre-Test Administration Pre-Test Administration Test Administration Test Administration Scoring Reporting Post-Test Administration Post-Test Administration Content and data reviews Test form construction Field testing Item banking & statistics Content exchange / interoperability Planning & blueprinting Item types Content development & universal design Learning standard alignment Administration planning & scheduling Registration, assignment, Form sampling Online infrastructure readiness assessment Pre-session planning (paper / online) & setup Alternate form assignment Test form delivery Platform (paper, online, mobile) presentation Item content & tools Adaptive testing Response collection Proctoring controls Form content security Desktop security Accessibility Testing anomalies Computer scoring Professional scoring Algorithmic (AI) scoring Portfolio scoring Sub test / strand scoring Attemptedness Performance levels Scaling / norming Growth scores Range finding Assessment Life Cycle Assessment Life Cycle Individual reporting Diagnostic reporting Informing & personalizing instruction Performance on standards Dashboard / summary reporting Aggregation / disaggregation Exchanging results / data Psychometric analysis Equating Score tables - scaling, norming Performance levels / cut scores Field test analysis Aligning results with curriculum / instruction Program and teacher effectiveness

2011 Strategic Context of IT Architecture Delivery Model SBAC IT ARCHITECTURE

2011 PROJECT APPROACH Goals and Capabilities Drivers and Tradeoffs Scenario Mapping Detailed Architecture Reqs Conceptual Models Logical Models Consolidate Library Validate Architecture Define Arch Review Process Execute Architecture Governance Model User Personas AnalysisEnvisioning and Decomposition Validation and Documentation Governance and OSS Strategy Execution Scenarios Define OSS Mgmt Process 0-3 Weeks3-7 Weeks7-10 Weeks10-12 WeeksJan > Ongoing Assist Vendor Selection and Oversight Monitor OSS Development Activities Coordinate QA Activities Technology Models

2011 Update on Progress 1.Analysis - Workshops a.Item Authoring, Item Banking, Digital Library and Test Creation b.Scoring, Distributed Scoring, Reporting, Portals/Dashboards, and Interoperability c.Test Creation, Test Delivery Platform, Adaptive Testing and Integration Framework 2.First and mid-term reviews

2011 Update on Progress 1.Validation and Documentation – Mid-Term Review a.Architecture and Technology a.High Level System Component Diagram b.Non Functional Requirements c.Build vs. Buy d.Application Development Model e.Architecture Deliverable Schedule

2011 Update on Progress 1.Validation and Documentation – Mid-Term Review a.User Scenarios b.User Goals and Initial Epics c.Concerns, Probability and Impact d.Initial Personas e.Logical Component Diagram f.Interoperability Diagrams

2011 Logical Component Diagram

2011 Component Interfaces

2011 Component Transport

2011 Next Steps Scenario Mapping to Component System Domain Definition Component Definition Deployment Definition Data Architecture Definitions Non-Functional Requirements Definition Security Requirements Technical Architecture Definition Server hardware and software requirements Networking requirements and diagrams Database, data storage and archiving requirements and approach Systems management and monitoring requirements and possible vendors Middleware and integration software requirements and possible vendors Security requirements and approach for applications, data, and end-user access

Measured Progress ©2011 Question and Answer

Q & A SIIA Resources:  CCSSO/SIIA Assessment Advisory Forum, 10/26/11 – Video and Presentations to be archived at  SIIA PARCC Webinar, 4/6/  SIIA Public Comments on RTTA  SIIA Blog, Balancing Technology Standardization and Innovation in Race to the Top Assessments, 1/31/  Getting Ready for CC Online Assessments, SIIA ETGF Session, 4/12/  SIIA Comments to US DoED Proposed RTTA Technology Standards Priorities, 11/4/

Thank you for joining us today!