1. Facilitator: Debbie Schraeder, ESU#3

2. Resources/ Recognition Nebraska Department of Education Nebraska Department of Education Pages, links, PPTs ESU 10/ Understanding NeSA Podcast ESU 6/Standards Format Explanation Jim Brink and Bill Menousek, ESU3tv

3. Agenda K-5: Facilitate Learning in Relation to: Standards Instructional Tool (SIT) - Math and Reading Glossaries Grades 2-5: Facilitate Learning in Relation to: Table of Specifications (TOS) Performance Level Descriptors (PLDs) Indicator Summary Reports Reference Sheets Practice Tests Grades K-1: Facilitate Learning in Relation to: Math Manipulative Ideas SMARTBoard Activities Guided Math

4. NUMBER SENSE CONCEPTS: Students will communicate number sense concepts using multiple representations to reason, solve problems, and make connections within mathematics and across disciplines. GEOMETRIC CONCEPTS AND MEASUREMENT CONCEPTS: Students will communicate geometric concepts and measurement concepts using multiple representations to reason, solve problems and make connections within mathematics and across disciplines. ALGEBRAIC CONCEPTS: Students will communicate algebraic concepts using multiple representations to reason, solve problems and make connections within mathematics and across disciplines. DATA ANALYSIS/PROBABILITY CONCEPTS: Students will communicate data analysis/probability concepts using multiple representations to reason, solve problems and make connections within mathematics and across disciplines. Nebraska Mathematics Standards Nebraska Mathematics Standards 4

5. K-12 Comprehensive Content Standard in each section reflects the broad learning standards for students related to mathematics. K-12 Comprehensive Content Standard in each section reflects the broad learning standards for students related to mathematics. NUMBER SENSE CONCEPTS: Students will communicate number sense concepts using multiple representations to reason, solve problems, and make connections within mathematics and across disciplines. GEOMETRIC/MEASUREMENT CONCEPTS: Students will communicate geometric concepts and measurement concepts using multiple representations to reason, solve problems, and make connections within mathematics and across disciplines. ALGEBRAIC CONCEPTS: Students will communicate algebraic concepts using multiple representations to reason, solve problems, and make connections within mathematics and across disciplines. DATA ANALYSIS/PROBABILITY CONCEPTS: Students will communicate data analysis/probability concepts using multiple representations to reason, solve problems, and make connections within mathematics and across disciplines. 5

6. Number Sense Concepts: -Number System -Operations -Computation -Estimation Geometric Concepts Measurement Concepts: -Characteristics -Coordinate Geometry -Transformations -Spatial Modeling -Measurement Algebraic Concepts: - Relationships -Modeling in Context -Procedures Data Analysis/Probability Concepts: -Display and Analysis -Predictions and Inferences -Probability Concept Organizers 6

7.  K-12 Comprehensive Content Standards Identify broad K-12 learning standards for students related to mathematics  Grade Level Standards Statements that identify what students know and be able to do by the end of each identified grade  Curricular Indicators Examples that further define what a student should learn at each specified grade level 7

8. 8

9. K-12 Comprehensive Content Standards Grade Level Standards Curricular Indicators Local Curriculum Lesson Plans Classroom Instructional Strategies Specific Strategies for Individual Students Determined by Local School Districts/ Classroom Teachers Determined by State Board Guidance for Districts, Schools, and Teachers Determined by State Board Required for all Standards Curriculum Instruction 9

10. 10

11. For your grade level: ◦ Think of vocabulary words with which your students struggle. ◦ Are they in the glossary? If so, does the definition match your understanding of the word? ◦ What is interesting? ◦ What do you have questions about?

12. Using NeSA Data

13. 13

14. Nebraska schools should use NeSA data to... Provide feedback to students, parents and the community Inform instructional decisions. Inform curriculum development and revision. Measure program success and effectiveness. Promote accountability to meet state and federal requirements. 14

15. Understanding NeSA Data o What is NeSA? o How do we access and interpret NeSA data? o How do we use NeSA data? 15

16. What is NeSA?

17. NeSA is... A criterion-referenced summative test. A measurement of the revised Nebraska Math Standards. A tool including 48 to 60 multiple-choice items. A test administered to students online OR paper/pencil. 17

18.  Four concepts measured: number sense geometric/measurement algebraic data analysis/probability  Grades 3 through 8, 11  Two sessions  Results will be made public 18

19. 19

20. NeSA is... Administered during the spring of the year. Based on Tables of Specification and Performance Level Descriptors. Built upon the best thinking of Nebraska educators, national experts, and a worthy partner – Data Recognition Corporation. 20

21. 21

22. Level 1Recall Level 1Recall Level 2Skill/Concept Level 2Skill/Concept Level 3Strategic Thinking Level 3Strategic Thinking Level 4EXTENDED THINKING Level 4EXTENDED THINKING 22

23. 23 Depth of Knowledge Focuses on content standard in order to successfully complete an assessment/standard task. Descriptive, not a taxonomy Not the same as difficulty

24. 24 Mechanism to ensure that the intent of the standard and the level of student demonstration required by that standard matches the assessment items (required under NCLB) Provides cognitive processing ceiling (highest level students can be assessed) for item development Why Depth of Knowledge?

25. The Depth of Knowledge is NOT determined by the verb, but the context in which the verb is used and the depth of thinking required. 25

26. Same verb—three DOK levels DOK 3- Describe a model that you might use to represent the relationships that exist within the rock cycle. (requires deep understanding of rock cycle and a determination of how best to represent it) DOK 2- Describe the difference between metamorphic and igneous rocks. (requires cognitive processing to determine the differences in the two rock types) DOK 1- Describe three characteristics of metamorphic rocks. (simple recall) 26

27. Remember… Depth of Knowledge (DOK) is a scale of cognitive demand. DOK requires looking at the assessment item/standard-not student work-in order to determine the level. DOK is about the item/standard-not the student. The context of the assessment item/standard must be considered to determine the DOK-not just a look at what verb was chosen. 27

28. 28 Depth of Knowledge Grade Level Practice

29. Performance Level Descriptors What are... 29

30. 30 Goals for Instruction

31. Accommodation Guides What are... 31

32. Where do we find the content components of NeSA? www.education.ne.gov/Assessment Tables of Specification Performance Level Descriptors Accommodations Guides Webb’s DOK documents 32

33. 33

34. ~NeSA Terminology~ PERFORMANCE LEVELS - three possible categories of student performance on NeSA NeSA Performance Levels Exceeds the Standards Meets the Standards Below the Standards 34

35. 35 Cut score processes: Contrasting Group Method – 400+ teachers Bookmark Method – 100+ teachers State Board of Education Reviewed  Examined results of both processes  Examined NAEP and ACT results for Nebraska  Made decisions within recommended range at public meeting How are performance levels determined?

36. NeSA (CRT) vs. NRT ? 36 --Differences-- Purposes: NeSA is intended to match and measure identified standards and instruction. NRT is not intended to measure any state’s standards. The intention is to compare students to each other. Item Development: NeSA items with exact match to the standards – NDE had to prove the match with an independent alignment study NRT – No standards to match – matches inherent and previous knowledge, enriched homes, pre-skills.

37. NeSA (CRT) vs. NRT ? 37 --Similarities— All the psychometric steps – standard setting (Bookmark, Angoff, Contrasting Group) Reliabilities – KR 20-21 / Inter-rater Reliabilities Descriptive Statistics (Item P-values, Dif-analysis) Administration: Both standardized – are generally administered the same way.

38. NeSA REPORTS Individual Student Report School Student Roster School Indicator Summary School Performance Level Summary District Reading Indicator Summary District Performance Level Summary District Report of School Performance 38

39. Questions for Consideration What can we learn from this report? Do we have other data to support these results? What are the implications of this report? 39

40. Individual Student Report 40

41. School Student Roster 41

42. School Indicator Summary 42

43. School Performance Level Summary 43

44. District Reading Indicator Summary 44

45. District Performance Level Summary 45

46. District Report of School Performance 46

47. How do we use NeSA data?

48. o Curriculum alignment process: Are the tested indicators in our curriculum? -- Where? When are they taught? How are they instructed? At what DOK (Depth of Knowledge) level? By whom? 48

49. Test preparation processes. Examine PLDs and Tables of Specification. Do our students have opportunity to learn the tested indicators? How are they performing on the indicators on a day-to-day basis? Are we assessing them locally? 49

50. What’s on the test? Find out! What math content is being assessed? How much of the test is comprised of computation, word problems, and open- response? Look at copies of released items to get an understanding of what is needed to be successful. Share them with your students so that they can get a sense of the time constraint. Final Tip: 50

51.  Practice Tests Practice Tests  Have our students used practice tests?  Are our students familiar with the testing tools? Reference sheets?  Are we familiar with appropriate accommodations? 51

52. 52 January 23 - February 10, 2012 - NeSA Writing Testing Window March 26-May 4, 2012 - NeSA Mathematics, Reading and Science Testing Window Dates to Keep in Mind

53. NeSA results ARE an important data source! When combined with other information, these data can support curricular, instructional, and learning support decision making. --It’s all about the Continuous Improvement Process! 53

54. Commit to data analysis as a continuous process, not an event. (Reeves, 2009)

55. 55 Data Sources EXPLORE/PLAN/ACTAIMSweb DIBELSGates MacGinitie Reading Theme TestsChapter Tests Measures of Academic ProgressDevelopmental Reading Assessments (DRAs) Curriculum-Based MeasurementTeacher Observation Running Reading RecordsStudent Projects Nebraska schools have many sources of data: Each assessment tool has a purpose and a role in the big picture of Continuous Improvement! Commit to data analysis as a continuous process, not an event. (Reeves, 2009)

56. 56 Purposes and Roles of Assessment (1) Diagnostic (2) Inform Instruction (3) Inform Stakeholder Groups (4) Comparison of Students Commit to data analysis as a continuous process, not an event. (Reeves, 2009)

57.  Mathematics Curriculum Mathematics Curriculum  NeSA-M NeSA-M  Mathematics Standards Mathematics Standards  deb.romanek@nebraska.gov deb.romanek@nebraska.gov  Debbie Schraeder, ESU#3 dschraeder@esu3.org 402-597-4865 57