An Introduction to Writing 3D-Science Assessment Entities Dr. Joshua B An Introduction to Writing 3D-Science Assessment Entities Dr. Joshua B. Smith American Institutes for Research

2019 Science Administration Grade 5: old assessment aligned to old content with embedded field test items aligned to three-dimensional standards. Grade 7: old assessment aligned to old content. Grade 8: stand-alone field test aligned to three-dimensional standards. High School EOCs in Biology and Chemistry for Grades 9-12

2020 Science Administration Plan Science Grade 5: new assessment aligned to three-dimensional standards. Science Grade 8*: new assessment aligned to three-dimensional standards. Science Grade 11*: new assessment aligned to three-dimensional standards. *pending rule change

items and alignment To create a test form that complies with a blueprint, the items on that form need to have an alignment. The items must be written such that they address a particular aspect of the science requirements that appear on the blueprint. In order to align items, we take guidance from published content standards that outline the specific aspects of science that are intended to be taught and assessed.

items and alignment Standards are usually pretty straightforward and by themselves provide much of what we need to craft an aligned item.

three-dimensional science standards are rather more complex

three-dimensional science standards The Framework for K-12 Science Education, which underpins the three-dimensional standards concept, is organized around three core dimensions of how people understand the world using science. The authors of The Framework recognized that science aligns with the same basic philosophy regardless of discipline - and that doing “the work of science” refers to applications of a finite number of activities and themes which, like the philosophy of science itself, cross topical disciplines.

three-dimensional science standards Science and Engineering Practice (SEP): the activities Scientific and Engineering Practices 1. Asking questions (for science) and defining problems (for engineering) 2. Developing and using models 3. Planning and carrying out investigations 4. Analyzing and interpreting data 5. Using mathematics and computational thinking 6. Constructing explanations (for science) and designing solutions (for engineering) 7. Engaging in argument from evidence 8. Obtaining, evaluating, and communicating information

three-dimensional science standards Three-dimensional standards, then, focus on the “doing” of science more than on what we think of as traditional science content (i.e., the observations of the world/universe at large and the hypotheses and theories that we devise to explain those observations, which is what one-dimensional standards usually focus on).

three-dimensional science standards So three-dimesional standards work to weave together these activities of making/understanding observations about the natural world with the observations themselves (or at least with specific observations that were determined to be good examples to use for the science practice in question).

three-dimensional science standards Disciplinary Core Ideas (DCIs): big ideas Physical Sciences PS1: Matter and its interactions PS2: Motion and stability: Forces and interactions PS3: Energy PS4: Waves and their applications in technologies for information transfer Life Sciences LS1: From molecules to organisms: Structures and processes LS2: Ecosystems: Interactions, energy, and dynamics LS3: Heredity: Inheritance and variation of traits LS4: Biological evolution: Unity and diversity Earth and Space Sciences ESS1: Earth’s place in the universe ESS2: Earth’s systems ESS3: Earth and human activity Engineering, Technology, and Applications of Science ETS1: Engineering design ETS2: Links among engineering, technology, science, and society

three-dimensional science standards In this spirit of trying to achieve that seamless weave then, each three-dimensional standard is a blend of one or two ”big ideas” from a science discipline, one of these scientific activities that are common to the doing of all sciences, and one of a number of broad themes that are found to cross scientific disciplinary boundaries.

three-dimensional science standards Cross-Cutting Concepts (CCCs): the themes Crosscutting Concepts 1. Patterns 2. Cause and effect: Mechanism and explanation 3. Scale, proportion, and quantity 4. Systems and system models 5. Energy and matter: Flows, cycles, and conservation 6. Structure and function 7. Stability and change

aligning to the CCCs

three-dimensional science standards You cannot divide up a three-dimensional standard into its components, teach and/or assess them separately, and truthfully assert that what you are doing is aligned to that three-dimensional standard. The intent is for the standard to be dealt with as a unit. So, classroom lessons or test questions that are built around one of these standards are effectively dually-aligned to science process and content.

three-dimensional science standards Because of this weave of three dimensions to create each standard, and because the central dimension is the practice, each of these standards is really a statement that describes an expectation of performance at the end of instruction rather than a statement of information to be remembered.

three-dimensional science standards These Performance Expectations (PEs) state what a student should know and be able to do at the end of the period of instruction, related to the blend of dimensions that make up that particular PE.

Performance Expectation: what students should know and be able to do at the end of the period of instruction Dimensions/Foundations Each PE represents the integration of three dimensions of science education: scientific and engineering practices (SEPs), disciplinary core ideas (DCIs), and crosscutting concepts (CCCs)

how we apply the practices

recap: one-dimensional standards One-Dimensional standards address facts or process (you have to dually-align items to address both). Most often these standards address facts.

recap: three-dimensional standards Three-dimensional standards address science facts and process at the same time. Each standard is built from one or two “big ideas” from a science discipline, one of these scientific activities that are common to the doing of all sciences, and one of a number of broad themes that are found to cross scientific disciplinary boundaries.

3-D Learning = Science Performance at the Intersection of the Three Dimensions Phenomenon- Driven 3D Student Performances 1. Standards 2. Instruction 3. Assessment 4. Instructional Materials 5. Professional Development Science and Engineering Practices Crosscutting Concepts Disciplinary Core Ideas

3-D Learning = Science Performance at the Intersection of the Three Dimensions Phenomenon- Driven 3D Student Performances 1. Standards 2. Instruction 3. Assessment 4. Instructional Materials 5. Professional Development Science and Engineering Practices Crosscutting Concepts Disciplinary Core Ideas

phenomena Phenomenon: a discrete observation/event about the world/universe that can anchor a 3D-aligned classroom lesson or assessment entity. A phenomenon is essentially something interesting about the world that relates to one or two big ideas, and which can be studied using one of the scientific practices around which the PEs are built. Phenomena are intended to anchor classroom discussions; the phenomenon sparks a question which kicks off the discussion – but - the entire discussion should be focused on addressing or explaining the phenomenon.

phenomena in student learning The desire is that there is a shift from: I’m going to provide you with the definition of evaporation. Then we’re going to discuss how evaporation operates. The desire is that there is a shift to: In the early morning on a sunny day, there is a large puddle of rainwater in the middle of a sidewalk in town. After noon, the puddle has disappeared. Essentially, here is a puzzling event or process that is related to evaporation, and which can spark questions about the underlying cause: How and why did this happen?

why phenomena?

phenomena Phenomena (e.g., a sunburn) are specific examples of something that is happening in the world/universe—an event or a specific example of a general process. Phenomena are not the explanations or the scientific terminology behind that specific example of what is happening. They are the start-point of the conversation and the focus to be explained. They aren’t the explanation (that’s what the discussion about the phenomenon is for).

phenomena

3D standards and phenomena in the classroom A classroom lesson that is built around three-dimensional standards will ideally be anchored on a phenomenon that is related to one or two core ideas, and which can be explained by students using a specific activity(s). The students conduct the activity to address the phenomenon. Any assessment entities attempting to parallel this kind of classroom instruction need to be similarly crafted…

the structure of the AIR clusters AIR clusters aligned to three-dimensional standards all include the same basic elements: One phenomenon (usually in a stimulus on the left side of the screen) Materials which augment the phenomenon (in the stimulus) A cluster task statement (short statement at the end of the stimulus materials which provides the student information about the the overall goal of the cluster) Individual interactions (usually on the right side of the screen opposite the stimulus)

the structure of the AIR clusters Each cluster is designed to engage the examinee in a grade-appropriate, meaningful scientific activity aligned to a specific standard. This task should be explicitly stated in the stimulus materials in a clear statement. Each cluster begins with a phenomenon, which anchors the entire cluster. The interactions within the cluster all address the phenomenon. The phenomenon and materials that augment it are usually set apart from the items in a stimulus. The interactions are engaged in sequence. Each interaction in the cluster is aligned to at least two of the three dimensions (SEP, DCI, CCC) and if possible all three.

the structure of the AIR clusters

the structure of the AIR clusters

the structure of the AIR clusters Stimulus Performance Expectation Cluster SEP Interaction 1 Part A, B, etc. Task Demand 1 DCI DCI SEP DCI CCC Interaction 2 Part A, B, etc. Task Demand 2 CCC Interaction 3 Part A, B, etc. SEP Task Demands 3 & 4 DCI Interaction 4 Part A, B, etc. SEP Task Demands 2 & 4 DCI CCC

AIR three-dimensional entities Clusters and Stand-Alone Items Stimuli include text/imagery/animations/simulations phenomenon & text phenomenon & text; graphics interaction several interactions are aligned to the stimulus, perhaps related to different parts stimulus stem text prompts text & animation Item cluster

the structure of the stand-alones

2019 Science Administration Grade 5: old assessment aligned to old content with embedded field test items aligned to three-dimensional standards. Grade 7: old assessment aligned to old content. Grade 8: stand-alone field test aligned to three-dimensional standards. High School EOCs in Biology and Chemistry for Grades 9-12

2020 Science Administration Plan Science Grade 5: new assessment aligned to three-dimensional standards. Science Grade 8*: new assessment aligned to three-dimensional standards. Science Grade 11*: new assessment aligned to three-dimensional standards. *pending rule change