Measuring What Matters: Technology & the Assessment of all Students Jim Pellegrino

Discussion Topics o Assessment and the National Educational Technology Plan o Research & Conceptual Foundations o Towards a New Generation of Technology- Enabled Assessments o Some Final Thoughts About Technology & the Assessment of All Students

NETP: What is it? o Response to Congressional mandate for five-year plan for educational uses of technology o Plan for transforming education with technology in response to urgent need to remain competitive in a global economy o Reflection of increased understanding of how to support learning and of growing capabilities enabled by technology

The Five Goals Productivity Teaching Infrastructure Learning Assessment

o Measure what matters o Improve learning in the moment; don’t just measure it after the fact o Use technology to measure complex competencies o Create electronic learning records, similar to electronic medical records o Aggregate and analyze data on a system level Assessment

Discussion Topics o Assessment and the National Educational Technology Plan o Research & Conceptual Foundations o Towards a New Generation of Technology- Enabled Assessments o Some Final Thoughts About Technology & the Assessment of All Students

Assessment as a Process of Reasoning from Evidence o cognition m theory or model of how students represent knowledge & develop competence in the domain o observations m tasks or situations that allow one to observe students’ performance o interpretation m method for making sense of the data observation interpretation cognition Must be coordinated!

Discussion Topics o Assessment and the National Educational Technology Plan o Research & Conceptual Foundations o Towards a New Generation of Technology- Enabled Assessments o Some Final Thoughts About Technology & the Assessment of All Students

Advantages of Technology for Science, Math & Literacy Assessment  Present authentic, rich, dynamic, interactive environments o Support access to collections of information, expertise o Present phenomena difficult or impossible to observe and manipulate in classrooms o Represent temporal, causal, dynamic relationships “in action” o Allow multiple representations of stimuli and their interactions (e.g., data generated during a process) o Allow overlays of representations, symbols, natural language o Allow student manipulations/investigations, multiple trials o Allow student control of pacing, replay, reiterate o Capture student responses during research, design, problem solving o Allow use or simulations of a range of tools and supports (internet, productivity, domain-based)

In the experiment that you just analyzed, the amount of alewife was set to 20 at the beginning. Another student hypothesized that the result might be very different if she started with a larger or smaller amount of alewife at the beginning. Run three experiments to test that hypothesis. At the end of each experiment record your data by taking pictures of the resulting graphs. After three runs, you will be shown your results and asked if it makes any difference if the beginning amount of alewife is larger or smaller than 20. Life Science Simulation

Examples of Formative Assessment with Feedback Diagnoser (FACET Innovations, LLC) Science Grades 6-12 Web-based Diagnoses “Facets” of student understanding Provides counter examples and additional lessons ASSISTments (Worcester Polytechnic Institute) Math Grades 4-10 Web-based Tutors students through assessment items Reports to teachers on 90 skills SimScientists (WestEd) Science Grades 6-8 Web-based Simulations of science phenomena used to assess students Reports to students and teachers

Features and Advantages Diagnostic Feedback and Coaching Formative Animations/Video Simulations Can show things impossible in P&P Multimedia Can assess skills like inquiry Natural Language Processing Immediate Scoring Detailed summative reports Reports for instruction Reporting

Discussion Topics o Assessment and the National Educational Technology Plan o Research & Conceptual Foundations o Towards a New Generation of Technology- Enabled Assessments o Some Final Thoughts About Technology & the Assessment of All Students

NETP Assessment Goal Our education system at all levels will leverage the power of technology to measure what matters and use assessment data for continuous improvement. o Design, develop, and adopt assessments that give students, educators, and other stakeholders timely and actionable feedback about student learning to improve achievement and instructional practices. o Build the capacity of educators and educational institutions to use technology to improve assessment materials and processes for both formative and summative uses. o Conduct research and development that explores how gaming technology, simulations, collaboration environments, and virtual worlds can be used in assessments to engage and motivate learners and to assess complex skills and performances embedded in standards. o Revise practices, policies, and regulations to ensure privacy and information protection while enabling a model of assessment that includes ongoing student learning data gathering and sharing for continuous improvement.