DAC Kick-Off Webinar CDE Assessment Unit August 26, 2014

Welcome and Agenda CDE Assessment Unit Staff Introductions 2014-2015 Assessment Calendar 2014-2015 Training CMAS and CoAlt: Science and Social Studies CMAS: PARCC English Language Arts and Mathematics CoAlt: Dynamic Learning Maps English Language Arts and Mathematics ACCESS for English Language Learners Colorado ACT Data Timeline NAEP and TIMSS

Meet the CDE Assessment Team Joyce Zurkowski Executive Director of Assessment Margo Allen Business Process Manager Meet the CDE Assessment Team Clayton Hollingshead Director of Assessment Administration Christina Wirth-Hawkins Director of Assessment Development Angela Landrum Formative Assessment Content Collaboratives Stephanie Boyd PARCC Sara Loerzel CMAS Administration & PearsonAccess Pam Sandoval NAEP & TIMSS Heather Villalobos Pavia ACCESS, ACT, Linguistic Accommodations Katie Schmidt CMAS Content Development Mira Monroe CoAlt, DLM, Special Ed Accommodations Jasmine Carey Psychometrics, ACT & DLM Data Collin Bonner Technology Sarah Underwood ACCESS and DLM Data Jessica Allen CMAS & PARCC Data

2014-2015 Assessment Calendar Assessment Grade Tentative Windows CMAS and CoAlt: Science and Social Studies 12 11/03/14 to 11/21/14 PARCC Performance-Based Assessment (PBA) 3-8, High School 3/09/15 to 4/03/15 4 and 7 (Social Studies) 5 and 8 (Science) 4/13/15 to 5/01/15 CoAlt: English language arts and math 3-11 4/13/15 to 5/15/15 PARCC End-of-Year Assessment (EOY) 4/27/15 to 5/22/15 ACCESS for ELLs® Reading, Writing, Speaking and Listening K-12 1/12/15 to 2/13/15  CO ACT Grade 11 Initial Test Date Tues., April 28, 2015 Make-up Test Date Tues., May 12, 2015

2014-2015 Training Assessment Description Dates (Tentative) CMAS: Science and Social Studies Score Interpretation Grade 12 Administration Technology September 8-19 (in-person) CoAlt: Science and Social Studies Administration End of September (online) All Accommodations September 29 and October 2 (online) CMAS: PARCC English Language Arts and Math January-February (in-person, combined) Elementary and Middle Administration CoAlt: DLM English Language Arts and Math Role Responsibilities August 21-September 8 (online) TBD (online) ACCESS for ELLs® October 20-30 CO ACT October 6 (online) Test Administration December 8 (online) Accommodations QA December 10 (online) Room Supervisor/Proctor Early February (online) When possible, webinars will be recorded for those who cannot attend.

CMAS and CoAlt Science and Social Studies Sara Loerzel (CMAS) Mira Monroe (CoAlt)

Performance Levels and Policy Claims Distinguished Command Students performing at this level demonstrate a distinguished command of the concepts, skills, and practices embodied by the Colorado Academic Standards for Science or Social Studies assessed at their grade level. They are academically well prepared to engage successfully in further studies in this content area. Strong Command Students performing at this level demonstrate a strong command of the concepts, skills, and practices embodied by the Colorado Academic Standards for Science or Social Studies assessed at their grade level. They are academically prepared to engage successfully in further studies in this content area. Moderate Command Students performing at this level demonstrate a moderate command of the concepts, skills, and practices embodied by the Colorado Academic Standards for Science or Social Studies assessed at their grade level. They will likely need academic support to engage successfully in further studies in this content area. Limited Command Students performing at this level demonstrate a limited command of the concepts, skills, and practices embodied by the Colorado Academic Standards for Science or Social Studies assessed at their grade level. They will likely need extensive academic support to engage successfully in further studies in this content area. 8

2014-2015 Assessment Calendar Assessment Grade Tentative Windows CMAS and CoAlt: Science and Social Studies 12 11/03/14 to 11/21/14 PARCC Performance-Based Assessment (PBA) 3-8, High School 3/09/15 to 4/03/15 4 and 7 (Social Studies) 5 and 8 (Science) 4/13/15 to 5/01/15 CoAlt: English language arts and math 3-11 4/13/15 to 5/15/15 PARCC End-of-Year Assessment (EOY) 4/27/15 to 5/22/15 ACCESS for ELLs® Reading, Writing, Speaking and Listening K-12 1/12/15 to 2/13/15  CO ACT Grade 11 Initial Test Date Tues., April 28, 2015 Make-up Test Date Tues., May 12, 2015

Science and Social Studies Assessments Elementary and Middle School Assessments Social Studies (4 & 7); Science (5 & 8) Administered for the first time this spring (2014) Approximately 250,000 students

Test Characteristics Distribution Targets Grade 4 Grade 7 Grade 5 Grade 8 Standard 1 25% 28% 36% Standard 2 27% 37.5% 30% Standard 3 23% 22% 34% Standard 4 (Nature of Science/Scientific Inquiry) (37.5%) (20%) Depth of Knowledge 1 15-20% 20-25% Depth of Knowledge 2 <55% <50% 45-50% Depth of Knowledge 3 25-30% 30-35% Performance Events /Simulations 32% 33% 38% Constructed Response /Selected Response 48%/52% 50%/50% 46%/54% 11

Operating Systems Used for CMAS Assessment Projections based on June 13, 2013 snapshot Spring Operating System Used by Students Chromebooks (Chrome OS) 10.3% 21.6% iPad (iOS) 5.4% 1.3% Mac OSX (10.6- 10.9.2) 12.4% 11.8% Windows XP 21.8% 7% Windows 7 48.4% 57.3% Windows other (Vista, Server, 8) 1.7% 1%                 Totals 100% 12

Challenges From First Operational Assessment Device, operating system, browser and test engine interactions New item types and tools Software updates Customer service and CenturyLink outages Bandwidth

Priorities for Future Administrations Improve TestNav Exit Process Streamline Process for Accommodation and Special Form Assignment Provide infrastructure trial resources that will allow districts to sufficiently test environments Refine PearsonAccess reporting features and status indicators Closely align PearsonAccess training and operational sites

New Content Standards New Assessments New Performance Standards (cut scores) 15

Standard Setting Educators from across the state participated Selected through open recruitment process by Pearson and CDE. Week of July 14th (August 11th) Performance Standards support the interpretation of student results Draft performance level descriptors were finalized during standard setting process State Board of Education Adopted performance level descriptors and cut scores at August meeting

Social Studies: Percent of Students at Each Performance Level Based on Final Cut Score Recommendations 17 17

Science: Percent of Students at Each Performance Level Based on Final Cut Score Recommendations 18 18

Scores and Uses Overall scale score and performance level Standard scale scores Selected Response and Constructed Response scale scores Prepared Graduate Competency and Grade Level Expectation raw score Item analysis

Elementary and Middle School Reporting Will be occurring in September District electronic files and pdf reports, including student data file (early to mid-September) School electronic files and pdf reports (mid-September) School student data file (mid-September) Hard copy individual student reports (mid- to late September) Item analysis reports (late September) CoAlt reports may lag about a week behind due to additional standard setting activities that occurred in August 20

Science and Social Studies Assessments High School Science and Social Studies (12) 2014-2015 Testing Window: November 3-21

Grade 12 Participants Students designated as 12th graders. Participating 12th graders typically will be students who are: Designated as 12th grader for the first time Part of School Year 2011-12 freshman cohort (2014-15 anticipated year of graduation) Part of School Year 2012-13 freshman cohort or later classified as 12th grader due to credit accumulation In their fourth year or less in high school

CMAS: High School Scheduling Each content area consists of 3 sections Option 1: Administer 3 sections to students in one sitting Option 2: Administer 1 section per day

CMAS: High School Timing CMAS: Science and Social Studies Section Administration Time for High School Estimated Time for Preparation Required Testing Time for All Students Additional Testing Time 10 minutes 55 minutes (15 minutes)

High School Critical Dates Manuals – September Procedures Manual (early-September) Test Administration Manual/Test Examiner Manual (mid-September) Data Supplement (late-September) Accommodations Supplement (late-September) PearsonAccess 8/29: High School ePAT with accommodations (TestNav 8) 9/8: PearsonAccess Training Site Available 9/22: Forms available to test technology environments (infrastructure trial) 10/9: Update participation counts for initial orders

Important Technology Dates Key Technology Dates (Tentative) DTC Webinars & Site Visits Upon Request 9/1 - 10/31 Practice Tests will be available in the PA Training Center for infrastructure trial 9/22 Quarterly Java Release 10/15 CMAS forms will be available for proctor caching No later than 10/27 DTC Technical phone support open 10/27 - 11/7 26

Upcoming TestNav-Dependent Software Release Dates Date of Release Action Required Firefox 31 July 22, 2014 None at this time Firefox 32 September 2, 2014 Chrome 38 Browser TBD (Aug 2014) Chrome OS 38 Chrome OS 39 TBD (Sept 2014) Java October, 15 2014 27

CMAS: Partnership for Assessment of Readiness for College and Careers (PARCC) English Language Arts and Mathematics Stephanie Boyd

2014-2015 Assessment Calendar Assessment Grade Tentative Windows CMAS and CoAlt: Science and Social Studies 12 11/03/14 to 11/21/14 PARCC Performance-Based Assessment (PBA) 3-8, High School 3/09/15 to 4/03/15 4 and 7 (Social Studies) 5 and 8 (Science) 4/13/15 to 5/01/15 CoAlt: English language arts and math 3-11 4/13/15 to 5/15/15 PARCC End-of-Year Assessment (EOY) 4/27/15 to 5/22/15 ACCESS for ELLs® Reading, Writing, Speaking and Listening K-12 1/12/15 to 2/13/15  CO ACT Grade 11 Initial Test Date Tues., April 28, 2015 Make-up Test Date Tues., May 12, 2015

Acronyms CBT Computer Based Test ELA (ELA/L) Name CBT Computer Based Test ELA (ELA/L) English Language Arts (English Language Arts/Literacy) EOY End of Year LEA Local Education Agency (District) PARCC Partnership for Assessment of Readiness for College and Careers PBA Performance Based Assessment PBT Paper Based Test 30

PARCC-Developed English Language Arts & Math Assessments Administration planned for the spring of 2015 The assessments have two components: Performance-based assessment administered 75% of the way through the school year End-of-year assessment administered 90% of the way through the year Grades included: English language arts: 3-11 Math: 3-8, with 3 assessments in high school

PARCC Assessment Design 75% of the year 90% of the year Performance-Based Assessment Extended tasks Applications of concepts and skills End-of-Year Assessment Innovative, computer-based items

Component -Content Area Current PARCC Design Component -Content Area # of Sessions Task Type Task Description Performance-Based Assessment: English Language Arts 3 Research simulation Literary analysis Narrative For each task, students will read one or more texts, answer several short vocabulary questions, and write an essay that requires them to draw evidence from the text(s) Performance-Based Assessment: Math 2 Short and extended-response questions Questions focus on conceptual knowledge and skills, and the mathematical practices of reasoning and modeling End-of-Year Assessment: English Language Arts 4-5 literary and informational texts Short-answer and vocabulary questions will be associated with each text End-of-Year Assessment: Math Primarily short answer and selected response questions Focused on conceptual knowledge, skills, and understandings

Secondary Math Two pathways are offered for high school math assessments: Algebra, Geometry, and Algebra II Integrated/Math I, Integrated/Math II, and Integrated/Math III

Secondary Math Options New high school mathematics assessment system has room for increased flexibility: Grade Assessment Options* 8 (3 options) 8th grade assessment or Algebra I/Integrated I 9 (4 options) Algebra I/Integrated I or Geometry/Integrated II 10 (6 options) Algebra I/Integrated I or Geometry/Integrated II or Algebra II/Integrated III 11 (4 options) Geometry/Integrated II or Algebra II/Integrated III 12 (2 options) Algebra II/Integrated III *Additional flexibility is being explored.

Testing Time The number of ELA items has been reduced https://www.parcconline.org/parcc-states-reduce-no-items-elaliteracy-portion-test Grades 3-5 - reduced by 2 passage sets (passage and questions) Grades 6-11 - reduced by 1 passage set PARCC is determining operational testing times based on field test performance. 36

Practice Materials A full set of practice tests will be available this fall/winter Practice tests for paper and pencil tests are being developed Practice tests are available at the following location: http://practice.parcc.testnav.com/#

CSAP/TCAP Writing – Middle School Imagine that your family is moving to a new place and your pet will not be allowed to live there. Write a letter to persuade the person in authority to allow your pet to live there. What kind of adult would you like to become? Write a paragraph to describe the kind of person you would like to be in ten years.

PARCC English Language Arts – Middle School You have read a website entry and an article and watched a video describing Amelia Earhart. All three include information that supports the claim that Earhart was a brave, courageous person. The three titles are: “The Biography of Amelia Earhart” “Earhart’s Final Resting Place Believed Found” “Amelia Earhart’s Life and Disappearance” (video) Consider the argument each author uses to demonstrate Earhart’s bravery. Write an essay that analyzes the strength of the arguments related to Earhart’s bravery in at least two of the three supporting materials. Remember to use textual evidence to support your ideas.

Preliminary Results How many students indicated that most to all of the math questions dealt with topics not covered in class? How many students indicated that most to all of the English language arts questions dealt with topics not covered in class? What percentage of students indicated that the math items were harder than their school work? What percentage of students indicated that the English language arts items were harder than their school work?

Capacity for Spring 2015 We know that elementary and middle schools can assess online; however, depending on number of devices: Multiple groups for each grade level Technology directed to assessment Current PARCC Design: 5 sections per student for the performance-based assessment 4 sections per student for the end-of-the-year assessment Option that is being considered: Offer mathematics assessments as paper-based assessments

CMAS: PARCC vs. Science/Social Studies PARCC has adopted PearsonAccess Next as the foundation to use Testnav 8. Some terms are changing: Student Data Upload (SDU) – is now Student Registration Import (SRI) This year both terms will be used, since we are on two different platforms Headphones will be required for the English language arts performance-based assessments. Students using text-to-speech will also require headphones unless they are tested in a separate environment.

Request for Early Test Window PARCC is finalizing testing window options. CDE expects to have a process similar to TCAP, under which districts will be able to request an early window The details are still being determined

CoAlt: Dynamic Learning Maps (DLM) English Language Arts and Mathematics Mira Monroe

2014-2015 Assessment Calendar Assessment Grade Tentative Windows CMAS and CoAlt: Science and Social Studies 12 11/03/14 to 11/21/14 PARCC Performance-Based Assessment (PBA) 3-8, High School 3/09/15 to 4/03/15 4 and 7 (Social Studies) 5 and 8 (Science) 4/13/15 to 5/01/15 CoAlt: English language arts and math 3-11 4/13/15 to 5/15/15 PARCC End-of-Year Assessment (EOY) 4/27/15 to 5/22/15 ACCESS for ELLs® Reading, Writing, Speaking and Listening K-12 1/12/15 to 2/13/15  CO ACT Grade 11 Initial Test Date Tues., April 28, 2015 Make-up Test Date Tues., May 12, 2015

What is DLM? Replaces CoAlt: Reading, Writing and Mathematics Grades 3 – 11 English Language Arts and Mathematics Consortia – Based at the Center for Educational Testing and Evaluation at KU Dynamic system adjusts to student responses 46

Terminology Educator Portal – a location in the KITE system Essential Elements (EEs) – selected nodes on the learning map The Colorado EEOs and DLM EEs are linked First Contact Survey – online questionnaire about the student for placement into the system KITE system – the platform used to house PD, training and deliver the assessment Learning Map – large web of skills that represent learning PNP – Personal Needs and Preferences Profile for how the student interacts with the system Testlet – a group of items, typically 3 – 8 47

Online Online for students KITE system May be taken off line, for students who are unable to access a computer Answers entered by the Test Examiner May require additional materials not provided by DLM or CDE KITE system 48

Online – Sample Item 49

Online – Sample Item 50

Sample/Familiar Text http://tarheelreader.org/find/ 51

Two Models Year End Required – for State accountability All students in each grade are assessed on the same Essential Elements (EE). Embedded Optional – district decision Educators have some choice of which Essential Elements (EE). Timing and frequency of assessment varies by district. Scores not used for State accountability Available beginning November 10 52

Possible Field Testing Opportunities Phase A: October 13 – 31 Phase B: November 10 – December 20 Phase C: January 5 – March 6 Braille: January 19 – February 6 (tentative) 53

Trainings and Professional Development Administration Training Do it this way and only this way Must be completed before testing Facilitated or self-directed Tests must be passed at 80% For an Educator to be linked to a District – the district must set them up in Educator Portal Professional Development Think about this stuff… consider your kids… make good choices Available in Educator Portal Not required 54

Next Steps Attend a DLM Webinar A recorded version will be available within two days of the live trainings DLM Webinar Date/Time (MST) Assessment Coordinator (DACs should attend) Friday, September 5 2:00 – 3:00 p.m. Monday, September 8 10:00 – 11:00 a.m. Data Steward (DACs should attend) Friday, August 29 Thursday, September 4 1:00 – 2:00 p.m. Technical Liaison Thursday, August 21 8:00 – 9:00 a.m. 55

DLM Administration Training Facilitated or Self-Directed – both require using modules in the Educator Portal Module 1 – Overview of the DLM System (Video: 15 min. plus quiz time) Module 2 – Test Security (Video: 6 min. plus quiz time) Module 3 – Accessibility for All Students (Video: 28 min. plus quiz time) Module 4 – How the Assessment Works (Video: 19 min. plus quiz time) Module 5 – Preparing for the Test (Video: 19 min. plus quiz time) Module 6 – Computer Delivered Testlets (Video: 24 min. plus quiz time) Module 7 – Teacher Administered Testlets (Video: 30 min. plus quiz time)

Heather Villalobos Pavia ACCESS for ELLs® Heather Villalobos Pavia

2014-2015 Assessment Calendar Assessment Grade Tentative Windows CMAS and CoAlt: Science and Social Studies 12 11/03/14 to 11/21/14 PARCC Performance-Based Assessment (PBA) 3-8, High School 3/09/15 to 4/03/15 4 and 7 (Social Studies) 5 and 8 (Science) 4/13/15 to 5/01/15 CoAlt: English language arts and math 3-11 4/13/15 to 5/15/15 PARCC End-of-Year Assessment (EOY) 4/27/15 to 5/22/15 ACCESS for ELLs® Reading, Writing, Speaking and Listening K-12 1/12/15 to 2/13/15  CO ACT Grade 11 Initial Test Date Tues., April 28, 2015 Make-up Test Date Tues., May 12, 2015

ACCESS for ELLs® Key Dates ACCESS Testing Window Monday, January 12 – Friday, February 13, 2015 Test Order Window, October 6 – November 12, 2014 If you do not need tests, still login to MetriTech and indicate zero Districts over-ordered last year Logistics Webinar, Thursday, October 9, 2015, 8:30 – 10:30 a.m. Tier Selection Support Webinar, Wednesday, October 15, 3:00 – 3:30 p.m. 59

ACCESS for ELLs® Key Dates (cont.) ACCESS Test Administration Trainings Pueblo: Monday, October 20, 2014 Aurora: Tuesday, October 21, 2014 Grand Junction: Tuesday, October 28, 2014 Greeley: Thursday, October 30, 2014 Return Materials to MetriTech, Thursday, February 19, 2015 60

Heather Villalobos Pavia Colorado ACT Heather Villalobos Pavia

2014-2015 Assessment Calendar Assessment Grade Tentative Windows CMAS and CoAlt: Science and Social Studies 12 11/03/14 to 11/21/14 PARCC Performance-Based Assessment (PBA) 3-8, High School 3/09/15 to 4/03/15 4 and 7 (Social Studies) 5 and 8 (Science) 4/13/15 to 5/01/15 CoAlt: English language arts and math 3-11 4/13/15 to 5/15/15 PARCC End-of-Year Assessment (EOY) 4/27/15 to 5/22/15 ACCESS for ELLs® Reading, Writing, Speaking and Listening K-12 1/12/15 to 2/13/15  CO ACT Grade 11 Initial Test Date Tues., April 28, 2015 Make-up Test Date Tues., May 12, 2015

ACT Testing Date: Tuesday, April 28, 2015 Make-up Date: May 12, 2015 Schools need to establish with ACT by October 17, 2014 Online Schools Voucher Delivery: December 1, 2014 NEW! ACT approved accommodations will be submitted online Can track approval progress Final date for submission: January 23, 2015 Only 1 test coordinator per school 63

Data Jessica Allen (CMAS and PARCC) Jasmine Carey (Psychometrics, ACT and DLM) Sarah Underwood (ACCESS and DLM)

Assessment Data Collections Assessment Collections have two Data Pipeline Activities Pre-Coded/Pre-ID Labels Student Biographical Data Review Training Materials Assessment Unit Data Pipeline Manual Annual Webinar 65

Assessment Collections – Pre-ID Labels Data for labels will be pulled directly from untagged Student Interchange DACs will be notified two weeks in advance of data being pulled 2014-2015 Tentative Pre-ID Timeline ACCESS for ELLs Early November CO ACT Mid January CMAS and CoAlt: Science and Social Studies - High School (Fall) Not Applicable Studies - Elementary and Middle School (Spring) Mid/Late January 2015 CMAS: PARCC English Language Arts and Math TBD CoAlt: DLM English Language Arts and Math 66

Uploading High School Demographic Data CMAS: Notify Jessica Allen at allen_j@cde.state.co.us by September 22 to use ACT demographic data from Spring 2014 OR Upload high school (12th grade) demographic data to PearsonAccess by October 9th for initial orders CoAlt:

Data Pipeline: Student Biographical Data (SBD) Collections 11/15/2018 Data Pipeline: Student Biographical Data (SBD) Collections The purpose of the Student Biographical Data (SBD) review is to provide districts with the opportunity to verify the accuracy of the demographic data submitted for each state assessment. 2014-2015 Tentative SBD Timeline ACCESS for ELLs March 26 – April 1 CO ACT Mid/Late June CMAS and CoAlt: Science and Social Studies - High School (Fall) January 2015 Studies -Elementary and Middle School (Spring) May 2015 CMAS: PARCC English Language Arts and Math TBD CoAlt: DLM ELA and Math 68

NAEP and TIMSS (selected schools) Pam Sandoval

Thank You!