Evidence-Centered Game Design Kristen DiCerbo, Ph.D. Principal Research Scientist, Pearson Learning Games Scientist, GlassLab.

Slides:



Advertisements
Similar presentations
Chapter 6 Working with Literature Zina OLeary. I not only use all the brains that I have, but all that I can borrow. Woodrow Wilson Zina OLeary (2009)
Advertisements

The Application of ECD to the Redesign of Advanced Placement Exams CCSSO June 2012 Maureen Ewing, Kristen Huff, Amy Hendrickson, Pamela Kaliski.
The Network of Dynamic Learning Communities C 107 F N Increasing Rigor February 5, 2011.
Objective Develop an understanding of Appendix B: CA ELD Standards Part II: Learning About How English Works.
© 2012 Autodesk Design Thinking: A Pathway to Innovation in Education Dr. Brian Donnelly Lecturer UC Davis School of Education, K-12 Education Consultant.
The Future of Assessment in the Digital Ocean Kristen DiCerbo, Ph.D. Principal Research Scientist, Pearson Distinguished Learning Games Scientist, GlassLab.
Common Core State Standards (CCSS) Nevada Joint Union High School District Nevada Union High School September 23, 2013 Louise Johnson, Ed.D. Superintendent.
© 2013 SRI International - Company Confidential and Proprietary Information Center for Technology in Learning SRI International NSF Showcase 2014 SIGCSE.
Robert J. Mislevy & Min Liu University of Maryland Geneva Haertel SRI International Robert J. Mislevy & Min Liu University of Maryland Geneva Haertel SRI.
MULTILINGUAL & MULTICULTURAL EDUCATION DEPARTMENT
SRI Technology Evaluation WorkshopSlide 1RJM 2/23/00 Leverage Points for Improving Educational Assessment Robert J. Mislevy, Linda S. Steinberg, and Russell.
University of Maryland Slide 1 May 2, 2001 ECD as KR * Robert J. Mislevy, University of Maryland Roy Levy, University of Maryland Eric G. Hansen, Educational.
University of Maryland Slide 1 July 6, 2005 Presented at Invited Symposium K3, “Assessment Engineering: An Emerging Discipline” at the annual meeting of.
5-1 McGraw-Hill/Irwin ©2002 by The McGraw-Hill Companies, Inc. All rights reserved. Chapter 5 Audit Planning.
U Iowa Slide 1 Sept 19, 2007 Some Terminology and Concepts for Simulation-Based Assessment Robert J. Mislevy University of Maryland In collaboration with.
Copyright ©2013 Pearson Education, Inc. publishing as Prentice Hall
Copyright ©2011 by Pearson Education, Inc. All rights reserved. Chapter 2: What are the Major Types of Social Research? MAN-10 Research Methods Instructor.
FERA 2001 Slide 1 November 6, 2001 Making Sense of Data from Complex Assessments Robert J. Mislevy University of Maryland Linda S. Steinberg & Russell.
Chapter 10.  Basic Functions  Insert Graphics, Audio/Video  Add Text  Create Links  Capture Brainstormed Ideas  Generate Outline  Organize Graphics,
Science and Engineering Practices
Copyright © 2010 Pearson Education, Inc. Leadership in Organizations 2-1 Chapter 2 Managerial Traits and Skills.
ADL Slide 1 December 15, 2009 Evidence-Centered Design and Cisco’s Packet Tracer Simulation-Based Assessment Robert J. Mislevy Professor, Measurement &
The Business Skills Handbook
Goals and Objectives.
The Use of Student Work as a Context for Promoting Student Understanding and Reasoning Yvonne Grant Portland MI Public Schools Michigan State University.
Pg 1 Design Research at GlassLab March 25 th, 2014.
Terry Vendlinski Geneva Haertel SRI International
Scientific Inquiry: Learning Science by Doing Science
Argumentation in Middle & High School Science Victor Sampson Assistant Professor of Science Education School of Teacher Education and FSU-Teach Florida.
Control technology. What and why? All the automatic electronic devices that regulate and control other devices. Control technology… …helps children understand.
Some Implications of Expertise Research for Educational Assessment Robert J. Mislevy University of Maryland National Center for Research on Evaluation,
A Role for Formalized Tools in Formative Assessment Bob Dolan, Senior Research Scientist, Pearson CCSSO NCSA | National Harbor |
The Depth of Knowledge (DOK) Matrix
SLB /04/07 Thinking and Communicating “The Spiritual Life is Thinking!” (R.B. Thieme, Jr.)
Copyright ©2015 Pearson Education, Inc.
Developing Assessments for and of Deeper Learning [Day 2b-afternoon session] Santa Clara County Office of Education June 25, 2014 Karin K. Hess, Ed.D.
Depth of Knowledge Assessments (D.O.K.) Roseville City School District Leadership Team.
Session Objectives Analyze the key components and process of PBL Evaluate the potential benefits and limitations of using PBL Prepare a draft plan for.
1 Orientation 101 General Education Program Student Learning Outcomes.
By Benjamin Newman.  Define “Cognitive Rigor” or “Cognitive Demand”  Understand the role (DOK) Depth of Knowledge plays with regards to teaching with.
Demonstrate employability skills required by business and industry.  The following elements should be integrated throughout the content of this course.
One Form of Argument… “Argument” in NGSS In science, the production of knowledge is dependent on a process of reasoning from evidence that requires a.
Teaching and Learning with Technology, 4e © 2011 Pearson Education, Inc. All rights reserved. Chapter 3 Designing and Planning Technology- Enhanced Instruction.
ACTion for Mathematics-ASPIRE. Background The math assessment was developed to reflect students’ knowledge and skill accumulation over time; capturing.
Performance Assessment of Critical Thinking: The Dolphin Dilemma Task Yigal Rosen, Ph.D. Sr. Research Scientist Pearson Knowledge Technologies.
Organizational Behavior 15th Ed
COMMON CORE CONTENT STANDARDS Module 1: Read and Know the Standard.
Activity 5a Systems of Professional Learning Module 3 Grades K–5: Supporting All Students in Writing and Research.
Funded by the Library of Congress.
Implementing the Common Core State Standards Monday, January 23rd - 4pm EST Deconstructing the Common Core Standards: Analyzing for Content, Level of Cognition.
Depth of Knowledge: Elementary ELA Smarter Balanced Professional Development for Washington High-need Schools University of Washington Tacoma Belinda Louie,
Assessing Math Looking closer at PARCC Task Types 2.
Inquiry Primer Version 1.0 Part 4: Scientific Inquiry.
New ELA Guidelines Shifts in ELA Common Core  Rise in Nonfiction Texts.  Content Area Literacy Close and careful reading of text  Increase Complexity.
Understanding Depth of Knowledge. Depth of Knowledge (DOK) Adapted from the model used by Norm Webb, University of Wisconsin, to align standards with.
Dr. Lois McKee- Assistant Principal Curriculum Mr. William Scales- Testing Coordinator The Florida Standards: What Every Parent Should Know Seminole High.
RCAS Elementary Literacy Bringing the Common Core State Standards to Life From Paper to Practice.
Introduction Social ecological approach to behavior change
Getting a Grip on Inquiry. "Independent Learning is that learning in which the learner, in conjunction with relevant others, can make the decisions necessary.
The Future of Assessment URSA 2016 Conference Cedar City, UT July 13-14, 2016.
Introduction Social ecological approach to behavior change
GCE Software Systems Development
BILC Seminar, Budapest, October 2016
Analyzing Performance Tasks: Turning Results Into Action
Technology Enhanced Items – Signal or Noise?
Critically Evaluating an Assessment Task
Setting Writing Goals in Science The Living Environment
Authentic Assessment Jacob K. Kelley.
Designing and Planning Technology-Enhanced Instruction
Unit 4 Review Answers.
Presentation transcript:

Evidence-Centered Game Design Kristen DiCerbo, Ph.D. Principal Research Scientist, Pearson Learning Games Scientist, GlassLab

Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.2

The ability to capture data from everyday events should fundamentally change how we think about learning and assessment. Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.3

Evidence-Centered Assessment Design What complex of knowledge, skills, or other attributes should be assessed? What behaviors or performances should reveal those constructs? What tasks or situations should elicit those behaviors? Mislevy, Steinberg, & Almond (2003)

Two Examples Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.5

Student Model - Aspire Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.6 Default Gateway Proficiency Understand the Purpose Obtain Navigate Devices Local vs. Remote Test Configuration Basic IP Addressing From Shute & Ventura, 2012

Task Model - Aspire Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.7

Evidence Model - Aspire Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.8

Student Model – Mars Generation One Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.9 LevelInterpretationExpression Achievement Preliminary Reason-1-I-A Identifies reasons people give to support a specific point Reason-1-E-A Generates at least one reason to support a specific point, in sentence form Foundational Reason-2-I-A Identifies supporting reasons or evidence in an argument and relates them to the point they support Reason-2-E-A Generates multiple reasons to support a point, and uses these reasons to counter others' argument in an engaging, familiar context Basic Reason-3-I-A Recognizes and explains the relationship between main and supporting points and keeps track of which evidence supports which point Reason--3-E-A Builds logical, hierarchically structured arguments by selecting and arranging reasons and evidence to support main and subsidiary points IntermediateReason-4-I-A Identifies specific points in a text that are vulnerable to objections and counterarguments Reason-4-E-A Creates simple critiques or rebuttals that critically provide summaries of or responses to other people’s arguments

Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.10 Task Model – Mars Generation One

Game mechanics Matching game mechanics and learning mechanics “The nature of the construct being assessed should guide the selection of relevant task…” Sam Messick, 1994 A game is built around a core mechanic A mechanic is a set of verbs that create a core loop If the core doesn’t match the competency, the competency isn’t integrated. Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.11 Find evidence Equip robot Battle robot Evaluate evidence Construct argument Critique argument

Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.12

Evidence – Mars Generation One Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.13 skill milestones / tasksstandards (CCSS)game phases/actions Evidence action #1 (specific actions that provide evidence) identify 1 piece of data that relates to specific argument CCSS.ELA-Literacy.WHST.6-8.1a, CCSS.ELA-Literacy.W.6.1akeeping, sorting, discarding constructing dual-core claim/data pair for argubots (google doc: Loop 1 Processes, Stage 1.3) identify >1 piece of data that relates to a specific argument CCSS.ELA-Literacy.WHST.6-8.1a; CCSS.ELA-Literacy.WHST.6-8.1b; CCSS.ELA-Literacy.W.6.1b sorting, organizing pre- battle constructing dual-core claim/data pair for argubots identify data as pro/con CCSS.ELA-Literacy.WHST.6-8.1a, CCSS.ELA-Literacy.W.6.1a sorting, discarding, (not) using in battle accuracy of using the relevancy switch/toggle during dual-core construction identify data as valid/invalid CCSS.ELA-Literacy.RH ; CCSS.ELA-Literacy.RST sorting, discarding, (not) using in battle relevancy switch/toggle during dual-core construction identify common arugmentation schemes CCSS.ELA-Literacy.RH ; CCSS.ELA-Literacy.RST sorting, interaction w NPC/computer, ogranizing pre-battle, battle appropriateness of argubot selected for dual-core

Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.15 Does it really get at what we think it does?

Cognitive Processes Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.16

0:002:004:006:008:0010:0012:0014:0016:0018:0020:0022:0024:0026:00 Pair 11 Pair 10 Pair 9 Pair 8 Pair 7 Pair 5 Pair 3 Pair 6 Pair 4 Pair 2 Pair 1 Problem Identification Solution Generation/ Implementation Solution Evaluation DiCerbo, Frezzo, & Deng, 2011

Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.18 Thank You

References DiCerbo, K. E. & Behrens, J. T. (2014). Impact of the digital ocean on education. London: Pearson. Available: DiCerbo, K. E., Frezzo, D. C., & Deng, T. (2011). Substantive validity of a simulation-based game. Research and Practice in Technology-Enabled Learning, 6(3), Available: pdf Shute, V. J. & Ventura, M. (2013). Stealth assessment in digital games. White paper for MIT series, published by the MacArthur Foundation. Available: For the masochist: Mislevy et al. (2014). Psychometric Considerations in Game-Based Assessment. Copyright © 2011 Pearson Education, Inc. or its affiliates. All rights reserved.19