Enriching Assessment of the Core Albert Oosterhof, Faranak Rohani, & Penny J. Gilmer Florida State University Center for Advancement of Learning and Assessment

Issues to Be Addressed Significant science competencies remain excluded from statewide assessments. Assessing samples of students could help expand types of science competencies assessed. Formative assessments are an integral part of the larger assessment picture. cala.fsu.edu/ies

Issues to Be Addressed Significant science competencies remain excluded from statewide assessments. Assessing samples of students could help expand types of science competencies assessed. Formative assessments are an integral part of the larger assessment picture. cala.fsu.edu/ies

Examples of Competencies That Go Unassessed Nature of Science Student can formulate a scientifically testable question(s) that relates to the context or data provided. Student can create a plan for carrying out a scientific investigation, including what, when, and how to measure variables. Student can make inferences and predictions and use the data to defend or refute [the] conclusion. cala.fsu.edu/ies

Examples of Competencies That Go Unassessed Life Science Student can observe and describe a local ecosystem. Student can determine potential limiting factors for specified populations in a local ecosystem. Student can explain why a species must continue to adapt when its environment changes in order for that species to continue to have an advantage. cala.fsu.edu/ies

Assessment Format Competencies such as those we listed can be assessed, just not with multiple-choice and other item formats that can be efficiently administered and scored. Efficient item formats, when well constructed, can measure important competencies, just not competencies such as those we listed. These more complex competencies require the use of performance assessments and similar formats. cala.fsu.edu/ies

Strategy We Are Investigating Teachers and the state education agency both develop assessments of complex competencies. The state’s assessments are administered to carefully selected samples to establish proficiency of students at the group level. Teachers establish the proficiency of individual students. Scores on assessments developed by teachers and the state agency are used to help cross-validate each other. cala.fsu.edu/ies

Participants cala.fsu.edu/ies

Specification Competency Evidence Example Task Scoring Plan for the Example Task Procedure for Creating Parallel Tasks Scoring Plan for the Parallel Tasks cala.fsu.edu/ies

Specification for Assessment #1 Formulating a Scientifically Testable Question Competency Student can formulate a scientifically testable question(s) that relates to the context or data provided. Evidence Students are provided a short list of variables that are related to a scientific topic or concept with which the students are familiar. The variables are listed without reference to how they could be observed, measured, or quantified. Approximately half of the variables are objectively measurable while the other variables are less objectively measurable. Students are asked to create and write down a scientifically testable research question that relates two of these variables in some way. To facilitate scoring, students also are asked to explain how each variable could be observed or measured. If a student selects a variable that is less objectively measurable, the student must describe a specific process to quantify observations of the variable that clearly is both objective and appropriate. A student’s performance is scored with respect to whether a scientifically testable question is provided that relates two of the listed variables. cala.fsu.edu/ies

Specification for Assessment #1 Formulating a Scientifically Testable Question Example Task In this exercise, you will create a scientifically testable question related to behaviors and other characteristics of flowers. Use two of the following seven variables when you create your scientifically testable question: –How beautiful a flower looks –How often insects (pollinators) visit a flower –How many flowers there are in a certain area –How much nectar a flower produces –How much a person enjoys the smell of a flower –How much time insects spend visiting a flower –How much familiarity a person has with a flower Choose two variables from the list that you think might have a relationship with one another. Only pick variables you (or another scientist) could actually observe or measure scientifically. Explain how each variable could be observed or measured. Write a scientifically testable question that relates your two variables to each other. cala.fsu.edu/ies

Specification for Assessment #1 Formulating a Scientifically Testable Question cala.fsu.edu/ies

Specification for Assessment #1 Formulating a Scientifically Testable Question Procedure for Creating Parallel Tasks cala.fsu.edu/ies

Specification for Assessment #1 Formulating a Scientifically Testable Question Scoring Plan for Parallel Tasks cala.fsu.edu/ies

Goals of Project cala.fsu.edu/ies

Goals of Project Develop example specifications relevant to science competencies not measurable with present statewide assessments. Determine if teachers and an external agent can independently develop comparable assessments based on these specifications. Using a generalizability theory, obtain estimates of score reliability. Examine the formative use of these assessments including teachers’ development of learning progressions. Note: Our project is not investigating the use of sampling, but instead is addressing issues highly relevant to the assessment model we propose. cala.fsu.edu/ies

Some Preliminary Results Related to Generalizability G-coefficients with Design of Rater : Agent cala.fsu.edu/ies

Some Preliminary Results Related to Comparability Frequency Distributions of Average Scores Assigned by Raters

Formative Assessment Black and Wiliams (1998) “We start from the self-evident proposition that teaching and learning must be interactive. Teachers need to know about their pupils’ needs – needs that are often unpredictable and that vary from one pupil to another.” Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2), cala.fsu.edu/ies

Formative Assessment Some Characteristics Occurs in real-time during learning to redirect instructional activities so as to improve learning. Is a package deal that involves substantially more than administering and scoring tests. Includes feedback, not only teacher to student, but also vice versa and student to student. Involves teacher and student activities that are responsive to this feedback. Proceeds through a deliberate learning progression. cala.fsu.edu/ies

Popham (2007) defines learning progressions as a “carefully sequenced set of building blocks that students must master en route to a more distant curricular aim. The building blocks consist of sub skills and bodies of enabling knowledge” (p. 83). Popham, J. W. (April 2007). The lowdown on learning progressions. Educational Leadership, 64(7), cala.fsu.edu/ies Formative Assessment Learning Progression

Heritage (2008) states that a “well-constructed learning progression presents a number of opportunities to teachers for instructional planning. It enables teachers to focus on important learning goals in the domain, centering their attention on what the student will learn rather than what the student will do (i.e., the learning activity)” (p. 5). Heritage, M. (2008). Learning progressions: Supporting instruction and formative assessment. Washington, DC: A paper published by the Council of Chief School Officers. cala.fsu.edu/ies Formative Assessment Learning Progression

Formative Assessment Worksheet

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), cala.fsu.edu/ies Formative Assessment Feedback

Interactions with Teachers Summer training workshop Follow-up workshop Review of teachers’ formative materials Class visitations Administration of external summative assessments cala.fsu.edu/ies

Enriching assessments is a critical component to enriching learning. cala.fsu.edu/ies

Center for Advancement of Learning and Assessment Florida State University