Summer 2012 DataCamp June/August, 2012

Everyone is using standards…

What Do We Ask Ourselves Before Building an Assessment Instrument? What do we want to measure? What information do we want to gather? How long do we want the assessment to take? How will we determine which standards we want to assess? How will we use the results?

Guiding Principles for Building Assessment Instruments A minimum of three items for every standard you want to measure. Spread items measuring an individual standard across the assessment. Range of overall assessment difficulty and for each standard measured. Begin assessment with easier items, build to more difficult and conclude with easier items.

Guiding Principles for Building Assessment Instruments Spread items with graphics across test. Readability levels for reading passages; genres needed. A minimum of five items per passage. Even spread of answer choices across assessment when possible; spread them out across the form. Check items for appropriateness to use online, paper, or both.

Use a blueprint to build validity Source: Ernie Bauer and Ed Roeber, “Technical Standards for Locally-Developed Assessments”, April 8, 2010 A grid is used to summarizes the content and format of the test. – The rows are the learning objectives – The columns are the level of cognitive complexity The cells list the types and numbers of items. You can place a margin that can be used to sum the total points.

Test Blueprint Depth of Knowledge RecallSkills/ConceptsStrategicExtended Target Totals #Points# # # # Learning Targets DOK Totals

Excellent Examples Webb (1997) Depth of Knowledge Recall Skills and Concepts Strategic Thinking Extended Thinking Anderson, & Krathwohl (2001) Revised Bloom’s Taxonomy

Depth of Knowledge Recall includes facts, definitions, terms, or a simple procedure, – “identify”, “recall”,“recognize” Skills and Concepts includes some processing beyond a habitual response – “classify”, “organize”, “compare” Strategic requires reasoning, planning, or using evidence – “justify”,“draw conclusion”, “explain” Extended Thinking requires complex reasoning, planning, developing, and thinking. – “evaluate”, “design”, “generalize”

Reverse Design Source: Ernie Bauer, and Jim Gullen, “Locally Developed Assessment: What Do the Results Mean?”, April 2010 Start with your items Group the items by content (GLCE/HSCE) – This will give you the rows for the blueprint Split the items in the content piles by DOK – This will give you the columns for the blueprint Create a blueprint document to check if this provides acceptable evidence for you.

Time Elements of an Assessment Calendar Source: White, S. H. (2005). “Beyond the Numbers: Making Data Work for Teachers and School Leaders”. Lead and Learn Press: Englewood, CO When will we administer the assessment? When will we collect the data? When will we disaggregate the data? When will we analyze the data? When will we reflect upon the data? When will we make recommendations? When will we make the decisions about the recommendations? When will we provide written documentation about the decisions? When will we share the data with other stakeholders?

“You Be George” A Formative Assessment Activity Imagine that you have just received some feedback on a recent science assessment. Using the information from your teacher, – Identify which learning targets you got right – Identify which learning targets you need to review – Identify which learning targets you need to keep practicing

Questions? Stan Masters Coordinator, Instructional Data Services Lenawee Intermediate School District 4107 N. Adrian Highway Adrian, Michigan (phone) (fax) Mike Husband Data Entry Specialist Lenawee Intermediate School District 4107 N. Adrian Highway Adrian, Michigan (phone) (fax) Data Warehouse webpage: