1. DAC Kick-Off Webinar
CDE Assessment Unit
August 26, 2014

2. Welcome and Agenda CDE Assessment Unit Staff Introductions
Assessment Calendar
Training
CMAS and CoAlt: Science and Social Studies
CMAS: PARCC English Language Arts and Mathematics
CoAlt: Dynamic Learning Maps English Language Arts and Mathematics
ACCESS for English Language Learners
Colorado ACT
Data Timeline
NAEP and TIMSS

3. Meet the CDE Assessment Team
Joyce Zurkowski
Executive Director of Assessment
Margo Allen
Business Process Manager
Meet the CDE Assessment Team
Clayton Hollingshead
Director of Assessment Administration
Christina Wirth-Hawkins
Director of Assessment Development
Angela Landrum
Formative Assessment Content Collaboratives
Stephanie Boyd
PARCC
Sara Loerzel
CMAS Administration & PearsonAccess
Pam Sandoval
NAEP & TIMSS
Heather
Villalobos Pavia
ACCESS, ACT, Linguistic Accommodations
Katie Schmidt
CMAS Content Development
Mira Monroe
CoAlt, DLM, Special Ed Accommodations
Jasmine Carey
Psychometrics,
ACT & DLM Data
Collin Bonner
Technology
Sarah Underwood
ACCESS and DLM Data
Jessica Allen
CMAS & PARCC Data

4. 2014-2015 Assessment Calendar Assessment Grade Tentative Windows
CMAS and CoAlt: Science and Social Studies 12
11/03/14 to 11/21/14
PARCC Performance-Based Assessment (PBA)
3-8, High School
3/09/15 to 4/03/15
4 and 7 (Social Studies) 5 and 8 (Science)
4/13/15 to 5/01/15
CoAlt: English language arts and math
3-11
4/13/15 to 5/15/15
PARCC End-of-Year Assessment (EOY)
4/27/15 to 5/22/15
ACCESS for ELLs®
Reading, Writing, Speaking and Listening
K-12
1/12/15 to 2/13/15 
CO ACT
Grade 11
Initial Test Date
Tues., April 28, 2015
Make-up Test Date
Tues., May 12, 2015

5. 2014-2015 Training Assessment Description Dates (Tentative)
CMAS: Science and Social Studies
Score Interpretation
Grade 12 Administration
Technology
September 8-19
(in-person)
CoAlt: Science and Social Studies
Administration
End of September
(online)
All
Accommodations
September 29 and October 2 (online)
CMAS: PARCC English Language Arts and Math
January-February
(in-person, combined)
Elementary and Middle Administration
CoAlt: DLM English Language Arts and Math
Role Responsibilities
August 21-September 8 (online)
TBD (online)
ACCESS for ELLs®
October 20-30
CO ACT
October 6 (online)
Test Administration
December 8 (online)
Accommodations QA
December 10 (online)
Room Supervisor/Proctor
Early February (online)
When possible, webinars will be recorded for those who cannot attend.

6. CMAS and CoAlt Science and Social Studies
Sara Loerzel (CMAS)
Mira Monroe (CoAlt)

8. Performance Levels and Policy Claims
Distinguished Command
Students performing at this level demonstrate a distinguished command of the concepts, skills, and practices embodied by the Colorado Academic Standards for Science or Social Studies assessed at their grade level. They are academically well prepared to engage successfully in further studies in this content area.
Strong Command
Students performing at this level demonstrate a strong command of the concepts, skills, and practices embodied by the Colorado Academic Standards for Science or Social Studies assessed at their grade level. They are academically prepared to engage successfully in further studies in this content area.
Moderate Command
Students performing at this level demonstrate a moderate command of the concepts, skills, and practices embodied by the Colorado Academic Standards for Science or Social Studies assessed at their grade level. They will likely need academic support to engage successfully in further studies in this content area.
Limited Command
Students performing at this level demonstrate a limited command of the concepts, skills, and practices embodied by the Colorado Academic Standards for Science or Social Studies assessed at their grade level. They will likely need extensive academic support to engage successfully in further studies in this content area. 8

9. 2014-2015 Assessment Calendar Assessment Grade Tentative Windows
CMAS and CoAlt: Science and Social Studies 12
11/03/14 to 11/21/14
PARCC Performance-Based Assessment (PBA)
3-8, High School
3/09/15 to 4/03/15
4 and 7 (Social Studies) 5 and 8 (Science)
4/13/15 to 5/01/15
CoAlt: English language arts and math
3-11
4/13/15 to 5/15/15
PARCC End-of-Year Assessment (EOY)
4/27/15 to 5/22/15
ACCESS for ELLs®
Reading, Writing, Speaking and Listening
K-12
1/12/15 to 2/13/15 
CO ACT
Grade 11
Initial Test Date
Tues., April 28, 2015
Make-up Test Date
Tues., May 12, 2015

10. Science and Social Studies Assessments
Elementary and Middle School Assessments
Social Studies (4 & 7); Science (5 & 8)
Administered for the first time this spring (2014)
Approximately 250,000 students

11. Test Characteristics Distribution Targets
Grade 4
Grade 7
Grade 5
Grade 8
Standard 1
25%
28%
36%
Standard 2
27%
37.5%
30%
Standard 3
23%
22%
34%
Standard 4
(Nature of Science/Scientific Inquiry)
(37.5%)
(20%)
Depth of Knowledge 1
15-20%
20-25%
Depth of Knowledge 2
<55%
<50%
45-50%
Depth of Knowledge 3
25-30%
30-35%
Performance Events /Simulations
32%
33%
38%
Constructed Response /Selected Response
48%/52%
50%/50%
46%/54% 11

12. Operating Systems Used for CMAS Assessment
Projections based on June 13, 2013 snapshot
Spring Operating System Used by Students
Chromebooks (Chrome OS)
10.3%
21.6%
iPad (iOS)
5.4%
1.3%
Mac OSX ( )
12.4%
11.8%
Windows XP
21.8% 7%
Windows 7
48.4%
57.3%
Windows other (Vista, Server, 8)
1.7% 1%
                Totals
100% 12

13. Challenges From First Operational Assessment
Device, operating system, browser and test engine interactions
New item types and tools
Software updates
Customer service and CenturyLink outages
Bandwidth

14. Priorities for Future Administrations
Improve TestNav Exit Process
Streamline Process for Accommodation and Special Form Assignment
Provide infrastructure trial resources that will allow districts to sufficiently test environments
Refine PearsonAccess reporting features and status indicators
Closely align PearsonAccess training and operational sites

15. New Content Standards New Assessments New Performance Standards (cut scores)15

16. Standard Setting Educators from across the state participated
Selected through open recruitment process by Pearson and CDE.
Week of July 14th (August 11th)
Performance Standards support the interpretation of student results
Draft performance level descriptors were finalized during standard setting process
State Board of Education Adopted performance level descriptors and cut scores at August meeting

17. Social Studies: Percent of Students at Each Performance Level Based on Final Cut Score Recommendations 17 17

18. Science: Percent of Students at Each Performance Level Based on Final Cut Score Recommendations18 18

19. Scores and Uses Overall scale score and performance level
Standard scale scores
Selected Response and Constructed Response scale scores
Prepared Graduate Competency and Grade Level Expectation raw score
Item analysis

20. Elementary and Middle School Reporting
Will be occurring in September
District electronic files and pdf reports, including student data file (early to mid-September)
School electronic files and pdf reports (mid-September)
School student data file (mid-September)
Hard copy individual student reports (mid- to late September)
Item analysis reports (late September)
CoAlt reports may lag about a week behind due to additional standard setting activities that occurred in August 20

21. Science and Social Studies Assessments
High School
Science and Social Studies (12)
Testing Window: November 3-21

22. Grade 12 Participants
Students designated as 12th graders. Participating 12th graders typically will be students who are:
Designated as 12th grader for the first time
Part of School Year freshman cohort ( anticipated year of graduation)
Part of School Year freshman cohort or later classified as 12th grader due to credit accumulation
In their fourth year or less in high school

23. CMAS: High School Scheduling
Each content area consists of 3 sections
Option 1: Administer 3 sections to students in one sitting
Option 2: Administer 1 section per day

24. CMAS: High School Timing
CMAS: Science and Social Studies Section Administration Time for High School
Estimated Time for Preparation
Required Testing Time for All Students
Additional Testing Time
10 minutes
55 minutes
(15 minutes)

25. High School Critical Dates
Manuals – September
Procedures Manual (early-September)
Test Administration Manual/Test Examiner Manual (mid-September)
Data Supplement (late-September)
Accommodations Supplement (late-September)
PearsonAccess
8/29: High School ePAT with accommodations (TestNav 8)
9/8: PearsonAccess Training Site Available
9/22: Forms available to test technology environments
(infrastructure trial)
10/9: Update participation counts for initial orders

26. Important Technology Dates
Key Technology Dates (Tentative)
DTC Webinars & Site Visits Upon Request
9/1 - 10/31
Practice Tests will be available in the PA Training Center for infrastructure trial
9/22
Quarterly Java Release
10/15
CMAS forms will be available for proctor caching
No later than 10/27
DTC Technical phone support open
10/ /7 26

27. Upcoming TestNav-Dependent Software Release Dates
Date of Release
Action Required
Firefox 31
July 22, 2014
None at this time
Firefox 32
September 2, 2014
Chrome 38 Browser
TBD (Aug 2014)
Chrome OS 38
Chrome OS 39
TBD (Sept 2014)
Java
October, 27

28. CMAS: Partnership for Assessment of Readiness for College and Careers (PARCC) English Language Arts and Mathematics
Stephanie Boyd

29. 2014-2015 Assessment Calendar Assessment Grade Tentative Windows
CMAS and CoAlt: Science and Social Studies 12
11/03/14 to 11/21/14
PARCC Performance-Based Assessment (PBA)
3-8, High School
3/09/15 to 4/03/15
4 and 7 (Social Studies) 5 and 8 (Science)
4/13/15 to 5/01/15
CoAlt: English language arts and math
3-11
4/13/15 to 5/15/15
PARCC End-of-Year Assessment (EOY)
4/27/15 to 5/22/15
ACCESS for ELLs®
Reading, Writing, Speaking and Listening
K-12
1/12/15 to 2/13/15 
CO ACT
Grade 11
Initial Test Date
Tues., April 28, 2015
Make-up Test Date
Tues., May 12, 2015

30. Acronyms CBT Computer Based Test ELA (ELA/L)
Name
CBT
Computer Based Test
ELA (ELA/L)
English Language Arts (English Language Arts/Literacy)
EOY
End of Year
LEA
Local Education Agency (District)
PARCC
Partnership for Assessment of Readiness for College and Careers
PBA
Performance Based Assessment
PBT
Paper Based Test 30

31. PARCC-Developed English Language Arts & Math Assessments
Administration planned for the spring of 2015
The assessments have two components:
Performance-based assessment administered 75% of the way through the school year
End-of-year assessment administered 90% of the way through the year
Grades included:
English language arts: 3-11
Math: 3-8, with 3 assessments in high school

32. PARCC Assessment Design
75% of the year 90% of the year
Performance-Based
Assessment
Extended tasks
Applications of concepts and skills
End-of-Year
Assessment
Innovative, computer-based items

33. Component -Content Area
Current PARCC Design
Component -Content Area
# of Sessions
Task Type
Task Description
Performance-Based Assessment: English Language Arts 3
Research simulation
Literary analysis
Narrative
For each task, students will read one or more texts, answer several short vocabulary questions, and write an essay that requires them to draw evidence from the text(s)
Performance-Based Assessment: Math 2
Short and extended-response questions
Questions focus on conceptual knowledge and skills, and the mathematical practices of reasoning and modeling
End-of-Year Assessment: English Language Arts
4-5 literary and informational texts
Short-answer and vocabulary questions will be associated with each text
End-of-Year Assessment: Math
Primarily short answer and selected response questions
Focused on conceptual knowledge, skills, and understandings

34. Secondary Math
Two pathways are offered for high school math assessments:
Algebra, Geometry, and Algebra II
Integrated/Math I, Integrated/Math II, and Integrated/Math III

35. Secondary Math Options
New high school mathematics assessment system has room for increased flexibility:
Grade
Assessment Options*
8 (3 options)
8th grade assessment or Algebra I/Integrated I
9 (4 options)
Algebra I/Integrated I or Geometry/Integrated II
10 (6 options)
Algebra I/Integrated I or Geometry/Integrated II or Algebra II/Integrated III
11 (4 options)
Geometry/Integrated II or Algebra II/Integrated III
12 (2 options)
Algebra II/Integrated III
*Additional flexibility is being explored.

36. Testing Time The number of ELA items has been reduced
Grades reduced by 2 passage sets (passage and questions)
Grades reduced by 1 passage set
PARCC is determining operational testing times based on field test performance. 36

37. Practice Materials
A full set of practice tests will be available this fall/winter
Practice tests for paper and pencil tests are being developed
Practice tests are available at the following location:

38. CSAP/TCAP Writing – Middle School
Imagine that your family is moving to a new place and your pet will not be allowed to live there. Write a letter to persuade the person in authority to allow your pet to live there. What kind of adult would you like to become? Write a paragraph to describe the kind of person you would like to be in ten years.

39. PARCC English Language Arts – Middle School
You have read a website entry and an article and watched a video describing Amelia Earhart. All three include information that supports the claim that Earhart was a brave, courageous person. The three titles are:
“The Biography of Amelia Earhart”
“Earhart’s Final Resting Place Believed Found”
“Amelia Earhart’s Life and Disappearance” (video)
Consider the argument each author uses to demonstrate Earhart’s bravery.
Write an essay that analyzes the strength of the arguments related to Earhart’s bravery in at least two of the three supporting materials. Remember to use textual evidence to support your ideas.

40. Preliminary Results
How many students indicated that most to all of the math questions dealt with topics not covered in class?
How many students indicated that most to all of the English language arts questions dealt with topics not covered in class?
What percentage of students indicated that the math items were harder than their school work?
What percentage of students indicated that the English language arts items were harder than their school work?

41. Capacity for Spring 2015
We know that elementary and middle schools can assess online; however, depending on number of devices:
Multiple groups for each grade level
Technology directed to assessment
Current PARCC Design:
5 sections per student for the performance-based assessment
4 sections per student for the end-of-the-year assessment
Option that is being considered: Offer mathematics assessments as paper-based assessments

42. CMAS: PARCC vs. Science/Social Studies
PARCC has adopted PearsonAccess Next as the foundation to use Testnav 8. Some terms are changing:
Student Data Upload (SDU) – is now Student Registration Import (SRI)
This year both terms will be used, since we are on two different platforms
Headphones will be required for the English language arts performance-based assessments. Students using text-to-speech will also require headphones unless they are tested in a separate environment.

43. Request for Early Test Window
PARCC is finalizing testing window options.
CDE expects to have a process similar to TCAP, under which districts will be able to request an early window
The details are still being determined

44. CoAlt: Dynamic Learning Maps (DLM) English Language Arts and Mathematics
Mira Monroe

45. 2014-2015 Assessment Calendar Assessment Grade Tentative Windows
CMAS and CoAlt: Science and Social Studies 12
11/03/14 to 11/21/14
PARCC Performance-Based Assessment (PBA)
3-8, High School
3/09/15 to 4/03/15
4 and 7 (Social Studies) 5 and 8 (Science)
4/13/15 to 5/01/15
CoAlt: English language arts and math
3-11
4/13/15 to 5/15/15
PARCC End-of-Year Assessment (EOY)
4/27/15 to 5/22/15
ACCESS for ELLs®
Reading, Writing, Speaking and Listening
K-12
1/12/15 to 2/13/15 
CO ACT
Grade 11
Initial Test Date
Tues., April 28, 2015
Make-up Test Date
Tues., May 12, 2015

46. What is DLM? Replaces CoAlt: Reading, Writing and Mathematics
Grades 3 – 11
English Language Arts and Mathematics
Consortia – Based at the Center for Educational Testing and Evaluation at KU
Dynamic system adjusts to student responses 46

47. Terminology Educator Portal – a location in the KITE system
Essential Elements (EEs) – selected nodes on the learning map
The Colorado EEOs and DLM EEs are linked
First Contact Survey – online questionnaire about the student for placement into the system
KITE system – the platform used to house PD, training and deliver the assessment
Learning Map – large web of skills that represent learning
PNP – Personal Needs and Preferences Profile for how the student interacts with the system
Testlet – a group of items, typically 3 – 8 47

48. Online Online for students KITE system
May be taken off line, for students who are unable to access a computer
Answers entered by the Test Examiner
May require additional materials not provided by DLM or CDE
KITE system 48

49. Online – Sample Item 49

50. Online – Sample Item 50

51. Sample/Familiar Text 51

52. Two Models Year End Required – for State accountability
All students in each grade are assessed on the same Essential Elements (EE).
Embedded
Optional – district decision
Educators have some choice of which Essential Elements (EE).
Timing and frequency of assessment varies by district.
Scores not used for State accountability
Available beginning November 10 52

53. Possible Field Testing Opportunities
Phase A: October 13 – 31
Phase B: November 10 – December 20
Phase C: January 5 – March 6
Braille: January 19 – February 6 (tentative) 53

54. Trainings and Professional Development
Administration Training
Do it this way and only this way
Must be completed before testing
Facilitated or self-directed
Tests must be passed at 80%
For an Educator to be linked to a District – the district must set them up in Educator Portal
Professional Development
Think about this stuff… consider your kids… make good choices
Available in Educator Portal
Not required 54

55. Next Steps Attend a DLM Webinar
A recorded version will be available within two days of the live trainings
DLM Webinar
Date/Time (MST)
Assessment Coordinator (DACs should attend)
Friday, September 5
2:00 – 3:00 p.m.
Monday, September 8
10:00 – 11:00 a.m.
Data Steward (DACs should attend)
Friday, August 29
Thursday, September 4
1:00 – 2:00 p.m.
Technical Liaison
Thursday, August 21
8:00 – 9:00 a.m. 55

56. DLM Administration Training
Facilitated or Self-Directed – both require using modules in the Educator Portal
Module 1 – Overview of the DLM System (Video: 15 min. plus quiz time)
Module 2 – Test Security (Video: 6 min. plus quiz time)
Module 3 – Accessibility for All Students (Video: 28 min. plus quiz time)
Module 4 – How the Assessment Works (Video: 19 min. plus quiz time)
Module 5 – Preparing for the Test (Video: 19 min. plus quiz time)
Module 6 – Computer Delivered Testlets (Video: 24 min. plus quiz time)
Module 7 – Teacher Administered Testlets (Video: 30 min. plus quiz time)

57. Heather Villalobos Pavia
ACCESS for ELLs®
Heather Villalobos Pavia

58. 2014-2015 Assessment Calendar Assessment Grade Tentative Windows
CMAS and CoAlt: Science and Social Studies 12
11/03/14 to 11/21/14
PARCC Performance-Based Assessment (PBA)
3-8, High School
3/09/15 to 4/03/15
4 and 7 (Social Studies) 5 and 8 (Science)
4/13/15 to 5/01/15
CoAlt: English language arts and math
3-11
4/13/15 to 5/15/15
PARCC End-of-Year Assessment (EOY)
4/27/15 to 5/22/15
ACCESS for ELLs®
Reading, Writing, Speaking and Listening
K-12
1/12/15 to 2/13/15 
CO ACT
Grade 11
Initial Test Date
Tues., April 28, 2015
Make-up Test Date
Tues., May 12, 2015

59. ACCESS for ELLs® Key Dates
ACCESS Testing Window
Monday, January 12 – Friday, February 13, 2015
Test Order Window, October 6 – November 12, 2014
If you do not need tests, still login to MetriTech and indicate zero
Districts over-ordered last year
Logistics Webinar, Thursday, October 9, 2015, 8:30 – 10:30 a.m.
Tier Selection Support Webinar, Wednesday, October 15, 3:00 – 3:30 p.m. 59

60. ACCESS for ELLs® Key Dates (cont.)
ACCESS Test Administration Trainings
Pueblo: Monday, October 20, 2014
Aurora: Tuesday, October 21, 2014
Grand Junction: Tuesday, October 28, 2014
Greeley: Thursday, October 30, 2014
Return Materials to MetriTech, Thursday, February 19, 2015 60

61. Heather Villalobos Pavia
Colorado ACT
Heather Villalobos Pavia

62. 2014-2015 Assessment Calendar Assessment Grade Tentative Windows
CMAS and CoAlt: Science and Social Studies 12
11/03/14 to 11/21/14
PARCC Performance-Based Assessment (PBA)
3-8, High School
3/09/15 to 4/03/15
4 and 7 (Social Studies) 5 and 8 (Science)
4/13/15 to 5/01/15
CoAlt: English language arts and math
3-11
4/13/15 to 5/15/15
PARCC End-of-Year Assessment (EOY)
4/27/15 to 5/22/15
ACCESS for ELLs®
Reading, Writing, Speaking and Listening
K-12
1/12/15 to 2/13/15 
CO ACT
Grade 11
Initial Test Date
Tues., April 28, 2015
Make-up Test Date
Tues., May 12, 2015

63. ACT Testing Date: Tuesday, April 28, 2015 Make-up Date: May 12, 2015
Schools need to establish with ACT by October 17, 2014
Online Schools Voucher Delivery: December 1, 2014
NEW!
ACT approved accommodations will be submitted online
Can track approval progress
Final date for submission: January 23, 2015
Only 1 test coordinator per school 63

64. Data Jessica Allen (CMAS and PARCC)
Jasmine Carey (Psychometrics, ACT and DLM)
Sarah Underwood (ACCESS and DLM)

65. Assessment Data Collections
Assessment Collections have two Data Pipeline Activities
Pre-Coded/Pre-ID Labels
Student Biographical Data Review
Training Materials
Assessment Unit Data Pipeline Manual
Annual Webinar 65

66. Assessment Collections – Pre-ID Labels
Data for labels will be pulled directly from untagged Student Interchange
DACs will be notified two weeks in advance of data being pulled
Tentative Pre-ID Timeline
ACCESS for ELLs
Early November
CO ACT
Mid January
CMAS and CoAlt: Science and Social
Studies - High School (Fall)
Not Applicable
Studies - Elementary and Middle School (Spring)
Mid/Late January 2015
CMAS: PARCC English Language Arts and Math
TBD
CoAlt: DLM English Language Arts and Math 66

67. Uploading High School Demographic Data
CMAS:
Notify Jessica Allen at by September 22 to use ACT demographic data from Spring 2014 OR
Upload high school (12th grade) demographic data to PearsonAccess by October 9th for initial orders
CoAlt:

68. Data Pipeline: Student Biographical Data (SBD) Collections
11/15/2018
Data Pipeline: Student Biographical Data (SBD) Collections
The purpose of the Student Biographical Data (SBD) review is to provide districts with the opportunity to verify the accuracy of the demographic data submitted for each state assessment.
Tentative SBD Timeline
ACCESS for ELLs
March 26 – April 1
CO ACT
Mid/Late June
CMAS and CoAlt: Science and Social
Studies - High School (Fall)
January 2015
Studies -Elementary and Middle School (Spring)
May 2015
CMAS: PARCC English Language Arts and Math
TBD
CoAlt: DLM ELA and Math 68

69. NAEP and TIMSS (selected schools)
Pam Sandoval

72. Thank You!