Using Results from a Practice Test in Data Director to impact SAT Instructional Decisions PSAT/NMSQT and SAT Practice Tests Emily McEvoy, Assessment Consultant, MISD November 17, 2015

KEY DELIVERABLES Receive a foundational understanding of the SAT Suite of Assessments and the key changes to the redesign Be able to interpret raw score conversions to a test score, section scores, sub-scores and cross section scores Learn to use the District Assessment and Item frequency reports in Data Director to identify the student successes and challenges Identify current instructional strengths and alignment to the SAT.

FOR OUR TIME TOGETHER Take responsibility for your own learning Support the learning of your colleagues If you need to take a call or have a side conversation, please step out in the hall If you have a question, please ask

For K–12 Students, Educators, and Parents

Not for MI

MI State Assessments

SPRING TEST DATES  11 th Grade  April 12 th - SAT w/ essay  April 26 th - Make-up date  April 13 th - WorkKeys  April 27 th – Make-up date  April 11 th – April 29 th (3 weeks) - M-Step  9 th & 10 th Grade  April 12 th or 13 th – PSAT  April 26 th or 27 th – Make-up date

PSAT AND SAT PRACTICE TESTS

Number Grid Responses Data Director recognizes ONLY 4 digit answers as correct. I.E. 300 and 0300 are not the same answer or 2 and 0002 Make sure to use the number grid directions attached to the practice tests in Data Director

FOCUS READ Individually read Section 2: Using SAT Suite of Assessments Scores and Reporting to Inform Instruction and Appendix: Standard Reports Explained In the reports, identify: √ = pertinent information ! = Important for teachers ? = Hmmm... I have questions about this

TABLE DISCUSSION Discuss your thoughts about the reports with your table. At your tables, select important elements from the reports at a building level. Make sure to take a break too!

Elements we can’t recreate as of now, Nov 17 th Instructional Planning Report Three performance groups related to the benchmarks Cohort Longitudinal Report Benchmark by Institution Report or Demographic Score Trends by Institution Report or Demographic

Important Elements from the Building Reports we can create using the Data Director Results Overall Test Score School Mean and SD County Mean and SD Section Scores Evidence-based Reading and Writing Mean and SD Math Mean and SD Test Scores Reading Mean and SD Writing Mean and SD Math Mean and SD Question Analysis Student responses Difficulty level Related cross-test scores and subscores * SD is Standard Deviation

How to find the converted scores in Data Director?

MEAN AND STANDARD DEVIATION

FIND THE STANDARD DEVIATION Plot all of the calculations on the bell curve. -2 , -1 , Mean, 1  and 2  Where on the scale of scores would the curve fall? Mean Macomb County PSAT Practice Test #1

ANOTHER EXAMPLE Plot all of the calculations on the bell curve. -2 , -1 , Mean, 1  and 2  Where on the scale of scores would the curve fall? Plot and answer the following for all scores: Describe the spread of scores. Where are most students falling on the scale? Mean Macomb County PSAT Practice Test #1

LUNCH

Important Elements from the Building Reports we can create using the Data Director Results Overall Test Score School Mean and SD County Mean and SD Section Scores Evidence-based Reading and Writing Mean and SD Math Mean and SD Test Scores Reading Mean and SD Writing Mean and SD Math Mean and SD Question Analysis Student responses Difficulty level Related cross-test scores and subscores * SD is Standard Deviation

In Data Director: Which pre-built reports could I use?

DISTRICT ASSESSMENT REPORT Prebuilt report in Data Director Using the Raw Scores

DISTRICT ASSESSMENT REPORT Page 1 and 2 Please NOTE: District means County There are items on both assessments that contribute to the scores for the Analysis of Science and His/S.S.

DISTRICT ASSESSMENT REPORT Page 1 and 2 Please NOTE: District means County There are items on both assessments that contribute to the scores for the Analysis of Science and His/S.S.

PSAT Practice Test #1 Total Number of Items Test Section, Subscore, Content Dimension and Item Difficulty The rest of the summaries are in your handout

SAT/PSAT Subscores, Domains and Content Dimensions: Reading Subscores Words in Context Command of Evidence Content Dimension Information and Ideas Rhetoric Synthesis Writing and Language Content Dimension Expression of Ideas (subscore) Development Organization Effective Language use Standard English Conventions (subscore) Sentence structure Conventions of Usage Conventions of Punctuation Math Domains (subscores) Heart of Algebra Problem Solving and Data Analysis Passport to Advanced Math Additional Topics (only SAT)

Identify a high and low performing Content Domain or Dimension using the District Assessment Report (pg. 1 and 2)

DISTRICT ASSESSMENT REPORT RESPONSE FREQUENCY Select the low performing items in the low performing content domain/dimension chosen.

DISTRICT ASSESSMENT REPORT RESPONSE FREQUENCY Select the high performing items in the high performing content domain/dimension chosen.

USING THE RESPONSE FREQUENCY In the low performing Content Domain or Dimension: Identify 10 questions within the content domain or dimension chosen where the majority of the students answered incorrect. This is when one of the other answer choices (not the correct answer) has a higher percent of students selecting it. Steps : Use the test booklet to review the test questions. Use the distractors to identify student misconceptions. Why did students choose B over D (correct answer)? Draw conclusions about why the students' performance was low on the item. Stay away from items students guessed on (responses will be evenly distributed among answer choices). Make sure to pick the best questions to analyze so that conclusions can be drawn.

USING THE RESPONSE FREQUENCY Do the same for the higher performing Content Domain or Dimension Identify 10 questions within the content domain or dimension chosen where the majority of the students answer correctly. What strengths are students able to demonstrate on these questions?

Share common patterns and themes across the low performing items vs. the higher performing items Next Steps : How will this analysis effect what we do in the classroom? instruction? Assessment structure/format? etc.