© 2003 SRI International Company Confidential Center for Technology in Learning 1 Supporting Geoscience Assessments with an Assessment Resource Management System (ARMS) Edys Quellmalz Geneva Haertel Daniel Zalles SRI International

© 2003 SRI International Center for Technology in Learning 2 Research on the Student Assessment Practices of Reform Programs Substantial effort and high costs involved in developing complex tasks (e.g., performance assessments) that provide better evidence of inquiry and problem-solving skills (CPRE, 1995; Pellegrino et al., 2001; Quellmalz, 1984) Disconnect between assessments useful for instruction and high-stakes accountability tests (Black & Wiliam, 1998; Stiggins, 2001) Need assessment literacy professional development (Stiggins, 2001) Need ongoing support for use of assessments aligned with standards and curricula (Darling-Hammond, 1995, 1996; Sparks & Loucks-Horsley, 1989)

© 2003 SRI International Center for Technology in Learning 3 Assessment Resource Management System (ARMS) Develop assessment resources that take advantage of the affordances of technology Collaborate with curriculum developers on embedded and external assessments Provide technical assistance, professional development, strategic consulting Develop innovative assessments in mathematics and science Conduct targeted research on alignment, design, cognitive analyses

© 2003 SRI International Center for Technology in Learning 4 Assessment Resource Management System (ARMS) Develop assessment resources that take advantage of the affordances of technology –On-line alignment guides –Digital collections of assessments –Development of innovative prototypes and collections –Tools to support on-line assessment delivery and rater training –On-line professional development on assessment literacy

© 2003 SRI International Center for Technology in Learning 5 Strategies for Development of Geoscience Student Assessments Develop preliminary atlas and strand maps for core geoscience concepts and inquiry Promote principled assessment design: align student outcomes, task models, evidence models Initiate collection of design patterns and exemplars for student assessments for core geoscience concepts and skills Develop assessment resources that take advantage of the affordances of technology Collaborate with course developers on embedded and external assessments Develop professional development guidelines for design of undergraduate student assessments Develop innovative assessments for geoscience Conduct targeted research on alignment, design, cognitive analyses

© 2003 SRI International Center for Technology in Learning 6 Assessment Resource Web sites: PALS

© 2003 SRI International Center for Technology in Learning 7 Assessment Resource Website: PALM

© 2003 SRI International Center for Technology in Learning 8 GLOBE Classroom Assessment Tools The GLOBE homepage provides access to: Links w/ standards Tools for development & revision of tasks Alignment protocols

© 2003 SRI International Center for Technology in Learning 9

© 2003 SRI International Center for Technology in Learning 10 Example of Performance Tasks Linked to National and State Standards Performance tasks in the PALS collection are linked to: Nat’l Science Education Standards (NSES) State Standards (IL, TX) Curriculum (FOSS, STC) Nat’l Council of Teachers of Mathematics (NCTM )

© 2003 SRI International Center for Technology in Learning 11 Search by NSES for Assessment Tasks

© 2003 SRI International Center for Technology in Learning 12 Addressing the Student Assessment Needs of Reform Programs

© 2003 SRI International Center for Technology in Learning 13 Search by State Science Standards for Assessments

© 2003 SRI International Center for Technology in Learning 14 Search by Curriculum Module for Assessment Tasks

© 2003 SRI International Center for Technology in Learning 15 PALS Standards-based Search Results

© 2003 SRI International Center for Technology in Learning 16 PALS Task Home Page Pick one of the sample assessment tasks from the online collection

© 2003 SRI International Center for Technology in Learning 17 Student Booklet Review individual performance assessment tasks…

© 2003 SRI International Center for Technology in Learning 18 Rubric Study the rubrics

© 2003 SRI International Center for Technology in Learning 19 Scored Work …Student work samples …

© 2003 SRI International Center for Technology in Learning 20 Technical Quality …Technical quality data from large scale administration

© 2003 SRI International Center for Technology in Learning 21 Online Training Online rater training perfects scoring techniques…

© 2003 SRI International Center for Technology in Learning 22 TPD Step by Step instruction

© 2003 SRI International Center for Technology in Learning 23 Adapt Analyze exemplars

© 2003 SRI International Center for Technology in Learning 24 Template to Create Use templates to create new tasks

© 2003 SRI International Center for Technology in Learning 25 Online Simulation Use online manipulatives or mind tools in investigations

© 2003 SRI International Center for Technology in Learning 26 Predator and Prey Performance Assessment

© 2003 SRI International Center for Technology in Learning 27 Collect information, take notes, and cite sources Data / information to collect: When does the hare population start to go down? What are reasons that can cause it to go down? » Canadian lynx » The lynx and hare » Predator-prey cycles Notes Lynx mostly eat hares. They don’t eat much else. If there aren’t many hares, they start to get hungry. Declarative Knowledge – Access & Organize Information / Data – identify required domain information search multiple representational formats

© 2003 SRI International Center for Technology in Learning 28

© 2003 SRI International Center for Technology in Learning 29 Use the Modeling tool – Observe population trends Your Web research indicated that although lynx depend on hares for food, introducing more lynx might decrease the hare population too quickly. But how quickly is too quickly? Dr. Kloss would like to use a modeling program to predict what will happen to the hare and lynx populations if you added more lynx to the parks. Once you’ve used the modeling tool, record your results below. Size of hare population if lynx are NOT added to the park: increase a lot increase a little decrease a little decrease a lot Size of hare population if lynx ARE added to the park: increase a lot increase a little decrease a little decrease a lot Analyze & Interpret Information / Data – use modeling tools to investigate, compare, test

© 2003 SRI International Center for Technology in Learning 30

© 2003 SRI International Center for Technology in Learning 31 Create a presentation Dr. Kloss would like you to prepare a short presentation (3 pages / slides) to the park service with your recommendations what to do about about the hare and lynx populations. Be sure your presentation is clearly organized and includes: A statement of the problem Your group’s recommendation Information, data and explanations to support your recommendation Communicate – present findings appropriate for audience present supported argument / findings

© 2003 SRI International Center for Technology in Learning 32 Solar Power Performance Assessment: Overview New federal funds are now available to increase the use of solar energy from photovoltaic cells. States have asked your energy consulting firm for recommendations about applying for these funds. You will use a series of visualizations to collect information that will help you identify two states that would generate a relatively high annual electricity yield from solar panels. You will then create a presentation to one of the two states, explaining why they should apply for the federal funds. Module 1: ICT Strategies: Plan strategies & procedure Technology Use: Review ArcView tool operations; practice response save Module 2: ICT Strategies: Analyze & interpret data; Represent & transform data Technology Use: Use ArcView features Module 3: ICT Strategies: Critically evaluate data; Plan needed data Module 4: ICT Strategy: Communicate recommendation and evidence Technology Use: Cut & paste screen shots, word process

© 2003 SRI International Center for Technology in Learning 33 ICT SOLAR POWER PERFORMANCE ASSESSMENT Procedural Knowledge – operate tools Access & Organize – search multiple representational formats Analyze & Interpret – Use visualization tools to investigate, compare

© 2003 SRI International Center for Technology in Learning 34 Module 4 High Response

© 2003 SRI International Center for Technology in Learning 35 Car Crash Performance Assessment: Overview When a 1200 kg car traveling at 17 m/s collides head-on with a wall, it generates a force greater than 10 5 Newtons (kg-m/sec 2 ). Slowing the car before the final impact decreases this force. Movable barriers (that look like large barrels) can be filled with sand or water and used as crash barriers. The State Highway Commission wants you to design a crash barrier that meets the following safety constraints: (1) Acceleration at any point cannot exceed –65 m/s 2 (2) Use 1-5 barrels in the design. Module 1: ICT Strategies: Create a graph and review physics concepts Technology Use: Review Interactive Physics tool operations Module 2: ICT Strategies: Analyze & interpret data; Represent & transform data Technology Use: Design a system to minimize crash damage Module 3: ICT Strategies: Critically evaluate data; Plan needed data Technology Use: Compare crash barrier designs Module 4: ICT Strategy: Communicate recommendation and evidence Technology Use: Cut & paste screen shots, word process Module 5: ICT Strategy: Compare two representations Module 4: ICT Strategy: Compare data Technology Use: Edit/create graphical representation

© 2003 SRI International Center for Technology in Learning 36