National Academy of Engineering Frontiers of Engineering Education Learning Panel Sheri Sheppard Karl Smith Wendy Newstetter Barbara Olds National Academy of Engineering Frontiers of Engineering Education November 2009

Session Objectives Participants will be able to Describe key features of the Backward Design process – Content (outcomes) – Assessment - Pedagogy Elaborate on why some things you do work well in promoting student learning Apply a few new strategies for assessment and instruction to classroom practice

What’s up… Framing the panel: POV1: Learning takes Action (Karl) POV2: Learning takes Feedback (Barbara) POV3: Base Learning Systems on Design Principles (Wendy)

Overarching or Ancillary Goal Framework #1: Backward Course Design Overarching or Ancillary Goal (for a specific group of students) Skills (what students will need to be able to do in order to attain this goal) Content Class Format Assessment Format Syllabus Does your syllabus share with your students the thinking process that you followed to design the course?

Education as a “Cognitive Apprenticeship” Framework #2: Education as a “Cognitive Apprenticeship” Modeling “representations of practice” Scaffolding “providing support” “deliberate, planned feedback” Coaching “removing scaffolding with growth” Fading Collins, Brown, Newman, 1989, pg. 480-483

National Academy of Engineering Frontiers of Engineering Education Learning Takes Action Karl A. Smith Engineering Education – Purdue University Civil Engineering - University of Minnesota ksmith@umn.edu - http://www.ce.umn.edu/~smith/ National Academy of Engineering Frontiers of Engineering Education November 2009

CAP Design Process Flowchart Integrated Course Design (Fink, 2003) 1. Situational Factors 2. Learning Goals 3. Feedback and Assessment 4. Teaching/Learning Activities 5. Integration Initial Design Phase Context Content Assessment Pedagogy C & A & P Alignment? End Start Yes No Backward Design 7

Reflect on Your Experience (yeah! The red arrow is pointing at one of our undergrad researchers—Michelle Valeriano) 2 1

Student Engagement “Perhaps the strongest conclusion that can be made is the least surprising. Simply put, the greater the student’s involvement or engagement in academic work or in the academic experience of college, the greater his or her level of knowledge acquisition and general cognitive development” - Pascarella and Terenzini (1991, 2005) Frequency and quality of student-student and student-faculty interaction are most influential for college student’s academic development, personal development and satisfaction (Astin, 1993; Light, 1992) High-Impact Educational Practices (Kuh, 2008)

The American College Teacher: National Norms for 2007-2008 Methods Used in “All” or “Most” All – 2005 All – 2008 Assistant - 2008 Cooperative Learning 48 59 66 Group Projects 33 36 61 Grading on a curve 19 17 14 Term/research papers 35 44 47 http://www.heri.ucla.edu/index.php

Calls for evidence-based teaching practices Engaged Pedagogy January 13, 2009—New York Times http://www.nytimes.com/2009/01/13/us/13physics.html?em January 2, 2009—Science, Vol. 323 www.sciencemag.org Calls for evidence-based teaching practices

http://www.ncsu.edu/PER/scaleup.html

http://web.mit.edu/edtech/casestudies/teal.html#video

http://www.udel.edu/pbl/ http://clte.asu.edu/active/smith.htm

Book Ends on a Class Session

Book Ends on a Class Session Advance Organizer Formulate-Share-Listen-Create (Turn-to-your-neighbor) -- repeated every 10-12 minutes Session Summary (Minute Paper) What was the most useful or meaningful thing you learned during this session? What question(s) remain uppermost in your mind as we end this session? What was the “muddiest” point in this session? Table summarizes my perception of the shift. A version of this table is available in New Paradigms for Engineering Education -- FIE Conf proceedings 97 (avail on the www) One of the most significant changes that has occurred is the shift from "pouring in knowledge" to "creating a climate where learning flows among students and the professor"

Reflect on the session: Session Summary (Minute Paper) Reflect on the session: 1. Most interesting, valuable, useful thing you learned. 2. Things that helped you learn. 3. Comments, suggestions, etc Pace: Too slow 1 . . . . 5 Too fast Relevance: Little 1 . . . 5 Lots Instructional Format: Ugh 1 . . . 5 Ah 17

Q4 – Pace: Too slow 1 . . . . 5 Too fast (3.3) MOT 8221 – Spring 2009 – Session 1 Q4 – Pace: Too slow 1 . . . . 5 Too fast (3.3) Q5 – Relevance: Little 1 . . . 5 Lots (4.2) Q6 – Format: Ugh 1 . . . 5 Ah (4.4) 18

MOT 8221 – Spring 2008 – Session 1 Q4 – Pace: Too slow 1 . . . . 5 Too fast (3.1) Q5 – Relevance: Little 1 . . . 5 Lots (3.9) Q6 – Format: Ugh 1 . . . 5 Ah (4.2)

Informal Cooperative Learning (Book Ends on a Class Session) Physics Peer Instruction - Eric Mazur - Harvard – http://galileo.harvard.edu Richard Hake – http://www.physics.indiana.edu/~hake/ Chemistry Chemistry ConcepTests - UW Madison www.chem.wisc.edu/~concept Video: Making Lectures Interactive with ConcepTests ModularChem Consortium – http://mc2.cchem.berkeley.edu/ STEMTEC Video: How Change Happens: Breaking the “Teach as You Were Taught” Cycle – Films for the Humanities & Sciences – www.films.com Harvard Thinking Together, From Questions to Concepts Interactive Teaching in Physics & Interactive Teaching DVD: Promoting Better Learning Using Peer Instruction and Just-In-Time Teaching: Derek Bok Center www.fas.harvard.edu/~bok_cen/

It could well be that faculty members of the twenty-first century college or university will find it necessary to set aside their roles as teachers and instead become designers of learning experiences, processes, and environments. James Duderstadt, 1999 [Nuclear Engineering Professor; Dean, Provost and President of the University of Michigan]

Resources Integrated Design Approach Pellegrino – Rethinking and Redesigning Curriculum, Instruction and Assessment Fink, L. Dee. 2003. Creating significant learning experiences: An integrated approach to designing college courses. San Francisco: Jossey-Bass. (Notes: Fink, L. Dee. 2003. A Self-Directed Guide to Designing Courses for Significant Learning). Backward Design Process (Wiggins & McTighe): Smith, K. A., Douglas, T. C., & Cox, M. 2009. Supportive teaching and learning strategies in STEM education. In R. Baldwin, (Ed.). Improving the climate for undergraduate teaching in STEM fields. New Directions for Teaching and Learning, 117, 19-32. San Francisco: Jossey-Bass. Bransford, Vye and Bateman – Creating High Quality Learning Environments Active & Cooperative Learning Smith, K.A., Sheppard, S. D., Johnson, D.W., & Johnson, R.T. 2005. Pedagogies of engagement: Classroom-based practices. Journal of Engineering Education Special Issue on the State of the Art and Practice of Engineering Education Research, 94 (1), 87-102.

Learning Takes Feedback Barbara M. Olds Sheri Sheppard National Academy of Engineering Frontiers of Engineering Education November 2009

Types of Feedback Testing and grading Informal assessment methods Purposes Definitions Guidelines Use of learning objectives Informal assessment methods

Grade Referencing Norm-referenced – classify and rank students relative to others; a relative standard (grading on the curve) Criterion-referenced – measure student achievement relative to a defined set of knowledge and skills; an absolute standard (competence)

Types of Assessment Informal – assessment activities designed to give quick, often qualitative, feedback to students Formal – assessment activities, often quantitative, designed to collect data for assigning a grade

Think/Pair/Share Exercise Describe the worst exam situation of your academic career List characteristics of the exam you would have changed to improve it Share your thoughts with a neighbor Be prepared to share with the class

Good Testing and Assessment Practices Test on what you teach (learning objectives) Test what you want your students to do Write tests that facilitate learning as well as provide data for grades

Good Testing and Assessment Practices Use non-graded formative assessments Use other assessment measures to supplement tests Don’t grade on the curve if you want students to learn together actively

Learning Objectives Effective way to communicate course expectations to students Measurable objectives form the basis for creating and assessing assignments, exams, projects, etc.

Learning Objectives Write objectives using quantifiable action verbs (e.g. apply, describe, compute, demonstrate, analyze, evaluate) Avoid terms which have no concrete meaning (e.g. know, learn, appreciate, understand)

Learning Objectives Try to write some objectives which require higher order thinking (e.g. divergent or evaluative thinking)

Classroom Assessment Small-scale assessments conducted in college classrooms by discipline-based teachers to determine what students are learning in that class. -- Tom Angelo

Classroom Assessment Is learner-centered, teacher-directed, ongoing, and context-specific Does not require specialized training Can be used to monitor individual student misconceptions and difficulties or overall class progress

Example CA Methods background knowledge survey minute paper concept map journal others on the handout More in Angelo and Cross, Classroom Assessment Techniques, 1993

Some Suggestions Don’t try any technique that doesn’t seem right Don’t make CA a burden Choose techniques that seem likely to produce useful data Allow enough time to conduct the assessment including response to data

One-minute Paper What is the most important bit of knowledge you learned in this session? What do you still have questions about?

Base Learning Systems on Design Principles Wendy C. Newstetter Director of Learning Sciences Research Wallace H. Coulter Department of Biomedical Engineering Georgia Institute of Technology wendy@bme.gatech.edu National Academy of Engineering Frontiers of Engineering Education November 2009

Design principles for learning systems Sustained apprenticeship to cognitive practice of model-based reasoning Scaffold meta-cognitive practices-monitor, reflect, change, improve Deep contextualization of practice in real world then textbook problems Learning forward/”knowing with” Thematic chaining

Problem-driven learning rooms

Distributed cognitive system

Design/problem solving studio

Learning forward through thematic chaining Why do we model? How do we do it? What can it tell us compared to other approaches? H1N1 modeling and simulation Year one PBL course Year two PBL course Eng CS Conservation DifEQ’s Cell physiology Year three PBL course Drug design for flu Bio-systems modeling Year four PBL course H1N1 cell level/Simulink Capstone design