Ohio Achievement Tests in Social Studies 2007 Results

OAT in Social Studies State-Wide Conclusions: Student performance was consistent across all seven standards. Student performance was consistent across content, i.e., world history and American history. Students had greater success with multiple- choice items than constructed-response items. Constructed response items were problematic for most students, with few students earning the maximum points available.

OAT in Social Studies Drawing conclusions from one administration of an achievement test is difficult because: a limited number of test items shows few patterns; item difficulty may vary within a standard or benchmark; data regarding grade-level content is limited; no baseline data is available for comparison.

Benchmark Level Items A.right to vote B.right to a trial by jury C.freedom of the press D.freedom of assembly All items are aligned to benchmark-level concepts and understandings. Grade 8 Social Studies OAT Multiple-Choice Item: 4.Which right found in the Magna Carta is also included in the United States Constitution?

Benchmark Level Items Standard: Citizenship Rights and Responsibilities Grade 6-8 Benchmark: B. Identify historical origins that influenced the rights U.S. citizens have today. All items are aligned to benchmark-level concepts and understandings.

Benchmark Level Items Grade-Level Indicators: Grade Six 2.Compare the rights and responsibilities of citizens living under various systems of government. Grade Seven 2.Describe the rights found in the Magna Carta and show connections to rights Americans have today. Grade Eight 4.Show connections between the rights and responsibilities of citizenship including: a.Voting and staying informed on issues; b.Being tried by a jury and serving on juries; c.Having rights and respecting the rights of others. All items are aligned to benchmark-level concepts and understandings.

Benchmark Level Items All items are aligned to benchmark-level concepts and understandings. Grade 5 Social Studies OAT Extended Response Question: 6. The Park Department needs volunteers to help clean up the neighborhood creek. A flyer announcing the cleanup is shown: In your Answer Document, identify two things the volunteers can do to help the creek. Explain how these actions will help promote the common good. (4 points)

Benchmark Level Items Standard: Citizenship Rights and Responsibilities Grade 3-5 Benchmark A: Explain how citizens take part in civic life in order to promote the common good. Grade-Level Indicators: Grade Three 1.Describe how people help to make the community a better place in which to live including: a.Working to preserve the environment; b.Helping the homeless; c.Restoring houses in low-income areas; d.Supporting education; e.Planning community events; f.Starting a business. All items are aligned to benchmark-level concepts and understandings.

Benchmark Level Items Standard: Citizenship Rights and Responsibilities Grade 3-5 Benchmark A. Explain how citizens take part in civic life in order to promote the common good. Grade-Level Indicators: Grade Four 1.Describe the ways in which citizens can promote the common good and influence their government including: a.Voting; b.Communicating with officials; c.Participating in civic and service organizations; d.Performing voluntary service. All items are aligned to benchmark-level concepts and understandings.

Standard: Citizenship Rights and Responsibilities Grade 3-5 Benchmark A: Explain how citizens take part in civic life in order to promote the common good. Grade-Level Indicators: Grade Five 1.Explain how an individual acquires U.S. citizenship: a.Birth: b.Naturalization. Benchmark Level Items All items are aligned to benchmark-level concepts and understandings.

Benchmark Data Grade 8 Social Studies OAT Multiple-Choice Items: 1. Governments interact with each other in many different ways. Which document is a formal agreement between two nations? A. treaty B. charter C. constitution D. proclamation 12. In 1620, the Mayflower Compact created a basis for written laws in the New England colony of Plymouth. Why did the Pilgrims create the Mayflower Compact? A. to declare independence from England B. to ensure peace with American Indians C. to establish an orderly community D. to guarantee universal religious tolerance in the colony Multiple items testing understanding of the same benchmark can vary in difficulty, limiting the conclusions that can be drawn.

Benchmark Data Standard: Government Benchmark: A. Explain why people institute governments, how they influence governments, and how governments interact with each other. 83% of students answered multiple-choice item #1 correctly. 44% of students answered multiple-choice item #12 correctly. Both items were aligned to the same benchmark : Multiple items testing understanding of the same benchmark can vary in difficulty, limiting the conclusions that can be drawn.

Benchmark Data Grade 5 Social Studies OAT Multiple-Choice Item: 25.Which newspaper headline would give you information about why a town is having an election next week? A.Councilman Quits to Take Job in Private Industry B.Council Votes to Increase Property Taxes C.Mayor Takes Oath of Office D.Crime Rate Rises in City Multiple items testing understanding of the same benchmark can vary in difficulty, limiting the conclusions that can be drawn.

Benchmark Data Grade 5 Social Studies OAT Multiple-Choice Item: Multiple items testing understanding of the same benchmark can vary in difficulty, limiting the conclusions that can be drawn. Speaker 1: “The king is forcing us to pay taxes on tea.” Speaker 2: “The colonists should not have to pay taxes to the king.” Speaker 3: “Colonists should have the same rights as other subjects of the king.” Speaker 4: “The king is a tyrant, and we must separate from England.” 34. Four Americans in the colonial period are talking about the British king. Which speaker is stating a fact? A.speaker 1 B.speaker 2 C.speaker 3 D.speaker 4

Benchmark Data Standard: Social Studies Skills and Methods Benchmark: B. Use a variety of sources to organize information and draw inferences. 21% of students answered multiple-choice item #25 correctly. 70% of students answered multiple-choice item #34 correctly. Both items were aligned to the same benchmark : Multiple items testing understanding of the same benchmark can vary in difficulty, limiting the conclusions that can be drawn.

Constructed Response Data Grade 8 Social Studies OAT Short-Answer Item: 38. Jan Logan opposes a bill in her state legislature that would raise the speed limit on rural highways. She believes that a higher speed limit would lead to more auto accidents in her community. In your Answer Document, identify one method Jan could use to keep this bill from becoming a state law in a representative democracy, and describe how the method accomplishes its goal. (2 points) Student performance shows difficulty earning the maximum number of points available on constructed response items, implying a potential difficulty in describing and/or explaining connections.

Constructed Response Data Performance Data: The percent of public school students earning each score point for question 38 on the May 2007 Ohio Achievement Test: Percent at Each Score Point % 18% 2 pointsThe student correctly identifies one method Jan could use to keep this bill from becoming a state law in a representative democracy and describes how the method accomplishes its goal. 1 pointThe student correctly identifies one method Jan could use to keep this bill from becoming a state law in a representative democracy, but does not describe how the method accomplishes its goal. The description may be incorrect, vague or missing.

Facts vs. Concepts 3-5 History6-8 History9-10 History A.Construct time lines B.Describe cultural patterns evident today. C.Explain developments A.Interpret relationships B.Describe characteristics C.Describe characteristics D.Describe effects E.Explain causes and consequences F.Explain challenges G.Analyze causes and consequences A.Explain connections B.Explain effects C.Analyze reasons D.Connect developments E.Analyze connections F.Identify patterns and explain their significance Items may not go beyond the cognitive level of the benchmark to which they are aligned.

Facts vs. Concepts Grade 5 Social Studies OAT Multiple-Choice Item: 40. In Sunflower, Mississippi, a group held a meeting in support of a candidate who is running for mayor. What constitutional right allowed this meeting to occur? A.freedom of assembly B.freedom of the press C.the right to a jury trial D.the right to vote Items may not go beyond the cognitive level of the benchmark to which they are aligned. What is the cognitive level of this item?

Facts vs. Concepts Items may not go beyond the cognitive level of the benchmark to which they are aligned. Citizenship Rights and Responsibilities, Benchmark B: Identify rights and responsibilities of citizenship in the United States that are important for preserving a democratic government. Grade Five, Grade-Level Indicator 3: Explain the significance of the rights that are protected by the First Amendment including: a.Freedom of religion; b.Freedom of speech; c.Freedom of the press; d.Right of petition and assembly.

Facts vs. Concepts Two students answering the item correctly explain why they selected freedom of assembly: Student 1: Our teacher had us memorize all of the rights listed in the First Amendment by using the abbreviation RAPPS, which stands for religion, assembly, press, petition and speech. I figured that assembly was the only one that had to do with meetings, so I picked A. Items may not go beyond the cognitive level of the benchmark to which they are aligned.

Facts vs. Concepts Two students answering the item correctly explain why they selected freedom of assembly: Student 2: I remembered learning about our rights as citizens and how the Constitution protects them. Our teacher had us work in groups to create pictures that showed citizens using their rights. The group that did ‘freedom of assembly’ drew a picture of people holding a meeting. This is an important right because it means that the government cannot stop people from having meetings when they are peaceful. That’s why I picked A. Items may not go beyond the cognitive level of the benchmark to which they are aligned. Which student recalled a fact? Which student showed understanding of a concept?

OAT in Social Studies Suggestions for school and district improvement: Encourage and provide opportunities for vertical teaming to plan instruction toward benchmark-level understandings; Share best practices across districts and state-wide; Use item annotations to inform instruction; Engage students in analyzing items and scoring constructed response items;

OAT in Social Studies Suggestions for school and district improvement: Emphasize writing in the content areas; Integrate the use of technology into instruction; Encourage differentiated instruction and assessment; Evaluate time spent on social studies as compared to other content areas.