Designing Offline and Online Learning Experiences for NGA Success Andrea Smith, NPESC Regional Director

What will the tests look like? What will students be asked to do? How can I modify my assessments to prepare my students? How can I modify learning experiences to prepare my students? What updated information is now available?

 Test Blueprints  Sample items  Quality Rubrics

PARCC-Developed Assessments English language arts ­ Grades 3 – 8 ­ End of Year exams (3) Mathematics ­ Grades 3 – 8 ­ Alg I, Geom, Alg II Operational school year State-Developed Assessments Science ­ Grades 5, 8 ­ End of Course – Bio and Phys Science (2) Social Studies ­ Grades 4, 6 ­ End of Course – Am His. & Am Govt (2) Operational school year Ohio’s Next Generation Assessments

Grades/Subjects Design and Format 3-8 -OAA Grade 10- OGT Aligned to old standards New Alternative Assessment for Severe Cognitively Disabled Students 3-8 -OAA aligned to existing and new standards Grade 10- OGT aligned to existing and new standards District developed EOC exams Am Hist and Am Govt (SB 165) NGA Field Testing 3-8- NGA--ELA and Math 3-8 NGA– Soc Stud. And Science HS- NGA-- ELA 9,10,11 and Alg1,Geo and Alg2 HS- NGA--Bio, Phys Science,Am Hist and Am Govt Grade 10 - OGT aligned to new standards PSAT 10 th grade CCR – postponed until Fall 2015 Web-based Two part summative(PBA and EOC/EOY) Diagnostic and Mid Year- PARCC Off Year PBA- State Developed in Soc Studies and Science

End-of-Year Assessment Innovative, computer-based items Performance-Based Assessment (PBA) Extended tasks Applications of concepts and skills Performance-Based Assessment (PBA) will assess knowledge of material from first three quarters of the year. Following the PARCC model, the PBA will present a combination of discret items and tasks linked to stimuli that engage significant content aligned to the model curriculum. The sequence of items associated with the stimulus draws the student into deeper analysis and interpretation than might ordinarily be possible in a single item. End-of-Year Assessment (EOY) administered after approx. 90% of the school year. All items will be scored by the computer. (PBA) (EOY)

 The Performance-Based Assessment (PBA) will assess the student’s knowledge of material from approximately the first three quarters of the course, as specified in this document.  The assessment will consist of approximately items worth 20 points overall.  It will require students to engage with course content at a significant cognitive depth and a meaningful level of analysis.  Following the PARCC model, the PBA will present a combination of discrete items and tasks, or sets of items linked to stimuli that engage significant content aligned to the model curriculum.

 The End-of-Year Examination will cover the entire content of the course.  It will be administered as close as possible to the end of the course (after approximately 90% of the course has been completed).  All EOY assessment items will be scored by computer, making possible a very quick return of scores.

 Multiple Choice- 4 answer options  Short Answer  Graphic Response  Simulation Test Items- Hand Scored Short (SCR) and Extended Constructed Response (ECR)- A question or set of questions that require a detailed written response. Responses are scored using a rubric.

A Multiple-choice item consists of the following: 1. a brief statement that orients the student to the context of the question (optional). 2. a stimulus (document, data table, graphic, etc.) on which the question is based (optional). 3. a question. 4. four answer options. Test Items- Computer Scored

A Short-answer item consists of the following: 1. a brief statement that orients the student to the context of the question (optional). 2. a stimulus (document, data table, graphic, etc.) to which the question refers (optional). 3. a question or prompt. 4. a response area. The student types a response to answer the question.

A Graphic-response item consists of the following: 1. a brief statement that orients the student to the context of the question (optional). 2. a stimulus (document, data table, graphic, etc.) to which the question refers (optional). 3. a question or prompt. 4. a graphic-response interface on which the student manipulates objects using a computer mouse to create a response to the question. The response interface may be a map, a chart or graph, a picture or a diagram on which the student must position objects correctly.

Sample GRAPHIC RESPONSE ITEM : Grade 5 Science – Delaware test

A Simulation  an interactive animated graphic interface that simulates an investigative experiment or physical situation.  Information is displayed in the form of dynamic maps or illustrations, statistical tables, or charts and graphs.  Data inputs can be adjusted by the student to reflect changes in the experimental or situational inputs, and the graphics adjust themselves to account for the new information.  Simulations are accompanied by more than one of the other item types.  The simulation functions as an interactive stimulus. Test Items- Computer Scored

 Stickleback Evolution ck-evolution-virtual-lab ck-evolution-virtual-lab  Bridge Challenge e/challenge/index.html e/challenge/index.html  Disease Detectives e-detective.html e-detective.html

A Short Constructed Response item (SCR) consists of the following: 1. a brief statement that orients the student to the context of the questions (optional). 2. one or more stimuli (documents, graphics, data displays, etc.) to which the questions refer (optional). 3. a question or set of questions that require a detailed written response or responses. The responses are scored according to a rubric or set of rubrics that address multiple dimensions in the student work. An Extended Constructed Response item (ECR) contains the same components as the SCR but requires a more elaborated response

 Go to ODE’s homepage (  Search “State Developed Assessments”

Beginning paragraphs:  Strand, Topic, Content Statement  Expectations for Learning- What are students expected to be able to DO? Content Limits- What students need/do not need to know for this standard Stimulus Attributes and Response Attributes  HOW will this standard be tested?  What do they expect students to get wrong? Short Answer question possibilities

Use your grade level blueprint to find the answers to the following questions:  Total points on the PBA?  How many points is the EOC assessment worth?  How many short answer questions will students answer on the PBA?  How many Multiple Choice questions on the PBA?  How will questions be divided between the different content strands?  What content will not be assessed on the PBA? Use your grade level blueprint to find the answers to the following questions:  Total points on the PBA?  How many points is the EOC assessment worth?  How many short answer questions will students answer on the PBA?  How many Multiple Choice questions on the PBA?  How will questions be divided between the different content strands?  What content will not be assessed on the PBA?

 What are 3 new things you have learned from the blueprints?  What are 2 questions you now have?  What is 1 thing you will use right away?

PBA min/sessionEOY min/session Grades 3 - 8Up to 75Up to 60 9,10,11Up to 90 Performance-Based Assessment (March) End-of-Year Assessment (May) 1 session for Science 1 session/day per student 2 Testing Sessions/day school 20 day testing window. 1 sessions for Science 1 session/day per student 2 Testing Sessions/day school 20 day testing window.

How do we prepare for online? Dealing with the "yeahbuts" and "wecants " AAGH!!! We won't have enough computers We don't know the cut scores We haven't seen a practice test Kids can't keyboard in 3rd grade These questions look hard Did we mention not enough technology?

Engaging assessments Technology enhanced items allow for multiple answers, modeling of thinking, use of simulations, embedded video or sound Test questions will provide scaffolding for students Built in accommodations Immediate results on the End of Year tests Larger testing window What are the benefits?

Keyboarding Student Tech Skills – For NGA Assessments Cutting and Pasting Highlighting Using on-screen calculator (gr 6-11 only) Dragging and Dropping items Manipulating a graph Running a simulation to generate data Changing font size and background color Clicking on multiple correct answers Utilizing spreadsheets, documents

Field Test Yearly test timeline Innovative item types Sessions and amount of time Accommodations Vendor for Ohio’s Science and Social Studies NGA is A.I.R.

Field Tests (Spring 2014)Practice (Fall 2014)Test Invitation Voluntary Computer-based Teachers’ view Results will guide item development Provided to all teachers in all districts Voluntary Computer-based only Students can test out the technology Teachers get a feel for the content tested

29 Features for All Students Accessibility Features* Identified in advance Accommodations* * * Available to all participating students **For students with disabilities, English learners, and English learners with disabilities

Accessibility Features for All Students Audio Amplification Blank Paper (provided by test administrator) Eliminate Answer Choices Flag Items for Review General Administration Directions Clarified (by test administrator) General Administration Directions Read Aloud and Repeated (by test administrator) Highlight Tool Headphones Magnification/Enlargement Device NotePad Pop-Up Glossary Redirect Student to Test (by test administrator) Spell Checker Writing Tools

Accessibility Features Identified in Advance Answer Masking Background/Font Color (Color Contrast) General Masking Line Reader Tool Text-to-Speech for the Mathematics Assessments

Administrative Considerations for All Students Principals may determine that any student may require one or more of the following test administration considerations, regardless of the student’s status as a student with a disability or who is an English learner: Small group testing Frequent breaks Time of day Separate or alternate location Specified area or seating Adaptive and specialized equipment or furniture

PARCC’s Proposed Accommodations for Students with Disabilities

34 Content Area Presentation Accommodations ELA/Literacy Text-to-Speech or Video of a Human Interpreter for the ELA/Literacy Assessments, including items, response options, and passages* Braille Edition of ELA/Literacy Assessments (Hard-copy braille tests and refreshable braille displays for ELA/Literacy) Closed-Captioning of Multimedia Passages on the ELA/Literacy Assessments Descriptive Video Mathematics Video of a Human Interpreter for the Mathematics Assessments for a Student Who is Deaf or Hard of Hearing Braille Edition of Mathematics Assessments (Hard-copy braille tests for Mathematics) Both Content Areas Additional Assistive Technology (Guidelines available fall 2013) Tactile Graphics Video of a Human Interpreter for Test Directions for a Student Who is Deaf or Hard of Hearing Paper-and-Pencil Edition * See notes below

35 Content AreaResponse Accommodations ELA/Literacy Scribing or Speech-to-Text (i.e., Dictation/Transcription or Signing) for constructed responses on the English Language Arts/Literacy Assessments* Word prediction on the ELA/Literacy Performance- Based Assessment* Mathematics Calculation Device and Mathematics Tools* (on Non-calculator Sessions of Mathematics Assessments) Both Content Areas Additional Assistive Technology (Guidelines available fall 2013) Braille note-taker Scribing or Speech-to-Text (i.e., Dictation/Transcription or Signing) for the Mathematics assessments, and for selected response items on the English Language Arts/Literacy assessments * See notes below

 Students taking the Physical Science Field Test will have three additional tools: ◦ An online scientific calculator ◦ An online Periodic Table ◦ An online Reference Sheet  See Appendix A, B and C handouts

 Go to:  Click “For Students and Families”  Choose Student Training Site  Choose your grade level and take one of the short online training assessments.  Answer keys are available – elementary, middle school, or secondary

 / /

1. Complete the online sample items – HIGH SCHOOL TEST/CALCULATOR/LINE TRACKER/PERIODIC TABLE 1. For each science sample item: ◦ Classify the item: MC, SA, GR, SCR, ECR ◦ Discuss the question’s level of cognitive demand ◦ Determine the grade level standards to which this question aligns 1. Complete the online sample items – HIGH SCHOOL TEST/CALCULATOR/LINE TRACKER/PERIODIC TABLE 1. For each science sample item: ◦ Classify the item: MC, SA, GR, SCR, ECR ◦ Discuss the question’s level of cognitive demand ◦ Determine the grade level standards to which this question aligns

Cognitive demands ◦ Describe the cognitive expectations associated with a learning task, the thinking that goes along with the doing ◦ Are an integral part of teaching and learning science ◦ Are differentiated from one another by the way knowledge is used rather than by the degree of rigor associated with a particular demand; they are not hierarchical ◦ Are interdependent and in effective science instruction, seldom appear in isolation

Requires students to Solve science-based engineering or technological problems Within given scientific constraints Propose and critique solutions Analyze and interpret engineering and technological problems Anticipate effects of engineering or technological design Consider consequences and alternatives Integrate and synthesize scientific information

Requires students to use scientific inquiry to Ask questions Plan and conduct investigations Gather and organize data Think critically and logically about relationships between evidence and explanations Construct and analyze alternative explanations Communicate scientific arguments

Requires students to Use subject-specific knowledge to interpret and explain events, phenomena, and concepts using grade-appropriate scientific terminology and technical and mathematical knowledge Communicate with clarity, focus, and organization using investigative scenarios, real-world data, and valid scientific information

Requires students to Provide accurate statements about valid scientific facts, concepts, and relationships Provide rote responses Perform routine mathematical tasks

Visit ORC's Examining Cognitive Demands e-pub to learn more by listening to podcasts and reviewing examples.

 Simulation Graphic Response

“Bottling Honey” from NAEP (National Assessment of Educational Progress) asks.asp asks.asp Simulation

Page&childpagename=Colorado/coPALPLayou t_v2&cid= &pagename=coPA LPWrapper

ol/practice_items/index.shtmlhttp:// ol/practice_items/index.shtml

Grade 8 science – Delaware test Scoring Rubric To earn full credit (3 points), a student places  the plants and/or the trees in the Producer column (1 point) AND  the mice and/or the owls in the Consumer column (1 point) AND  the bacteria in the Decomposer column (1 point). NOTE: Full credit (3 points) requires all three organisms to be placed in appropriate columns. To earn partial credit, a student answers one or two parts correctly. Graphic Response

Scoring Rubric To earn full credit (3 points), a student places  the “ribosome” label in the purple box (1 point) AND  the “DNA” label in the red box (1 point) AND  the “transcription” label in the green box (1 point). Full credit (3 points) requires all three terms to be placed in the correct three boxes. To earn partial credit, a student answers one or two parts correctly. 10 th Grade biology – Delaware test Graphic Response

 Which item types are going to be the easiest to incorporate into your classroom?  Which item types will be the most challenging? Why?  How can you use the item types with your colleagues?  How can you use the item types with your students?  Which item types are going to be the easiest to incorporate into your classroom?  Which item types will be the most challenging? Why?  How can you use the item types with your colleagues?  How can you use the item types with your students?

 Provide evidence for selected responses  Design and analyze the data from a multi-step investigation  Write interpretations of data based on evidence  Apply what is learned through simulated investigation of authentic scenarios

How can I modify my assessments to prepare my students?

 What classroom experiences will be necessary for students to be successful on a variety of online assessments?  What types of off-line learning will you be planning for your students?  What types of on-line learning will you be searching for in order to better prepare your students? Considerations for Instructional Shifting

  ash.shtml ash.shtml  hemistry hemistry   

Quality Review Rubric for Science

 Share the following: ◦ Any online resources that support NGA-type assessment you are currently using that you didn’t hear about today. ◦ At least one of the websites you saw today you want to investigate further

How the tests will look? What the students will be asked to do? How we can modify our assessments to prepare our students? What will I share with my colleagues about…..

 Dr. Jodi Haney, BGSU  Andrea Smith, Regional Director