Assessment = Learning What evidence will you and your peers accept? Diane Ebert-May, Ph.D. Lyman Briggs School Botany and Plant Pathology Michigan State University

InnovationsChangeInquiryActive LearningFacultyStudentsInstitutions OutcomesAssessment Content/ Process Self-Efficacy Cooperative Learning toward necessitates by collaborate with supported by achieve develop use in-depth value in-depth e.g. drives necessitates Reasoning

Cooperative Groups  4 students per group  Person A, B, C, D in each group  First - read question/think about answer individually  Discuss: A with B  C with D  Form group consensus

Q1 - Engagement  Imagine I invited you to visit and review my course Introductory Biology to help me decide if I was practicing “active learning.” What criteria and dimensions of teaching would you look for in my class?  Oral report from groups

Biological Literacy for All  Utilize process of scientific inquiry to think creatively and formulate questions about real- world problems  Effectively communicate an understanding of and links among biological principles and concepts to peers and others  Gain confidence in ability to write about, criticize and analyze concepts in biology

Biological Literacy for All (2)  Reason logically and critically to evaluate information  Develop positive attitudes about the relevance of biology to their lives  Demonstrate positive interdependence and individual accountability within cooperative groups

NSF- Undergraduate Course and Curriculum Development Grant  Is it possible to implement strategies for active, inquiry-based learning and cooperative group interaction in large courses?  Do these strategies promote more effective learning by more students? – students in class meetings – 450 students in laboratory

Compared Two Approaches to Teaching  Traditional  Experimental

Who Asks Questions  Traditional:Lab manual/faculty  Experimental:Students

Inquiry Lab/Class Meeting  Traditional:Confirmatory  Experimental:Inquiry

Functional Cooperative Groups  Traditional:Not Often  Experimental:Often

Writing/Reading to Learn  Traditional:Not Often  Experimental:Often

TA Professional Development  Traditional: Superficial  Experimental: In-depth, on-going

Assessment  Traditional:Tests  Experimental:Multiple Forms –Aligned with student outcomes

Cooperative Groups: 4 Students Engage minutes Explore minutes Explain minutes Elaborate minutes Evaluate10 minutes Learning Cycle Model in Class Meetings

Experimental Design Covariates Pre-Test / Factor Scores Control LecturesExperimental / Control LecturesExperimental Lecture / Lab Control / Experimental LabsExperimental Labs Fall 1994Spring 1995Fall 1995 Self-Efficacy Self-EfficacySelf-Efficacy NABT NABTNABT Process Skills I, II Process Skills I, II Process Skills I, II TOSRATOSRATOSRA Independent Variables Intervention --- TA --- Gender --- Ethnicity --- Age (Lab, Lecture)

Q2 - Exploration  What evidence will you (and your peers) accept that indicates your students have learned and achieved the outcomes of your course?  Written group report

Self-Efficacy Subscales  Fac 1: Writing and critiquing; analytical skills ( .92)  Fac 2: Generalizability to other science courses ( .87)  Fac 3: Biological literacy ( .88)

* p <.01 (n =338) Self-Efficacy, Fall 94 Experimental Condition EXPCON Factor Scores* PosFac 1 PosFac 2 PosFac 3

NABT Process, Fall 1994 p <.01 (n =356) Lecture ExperimentalControl

NABT Content  No difference in scores between traditional and control groups

Model with 700 students  Two 350 students each  Taught back to back on schedule  T, Th - 75 minute periods  Team taught - 2 faculty as team partners in both sections

Number of Absences

B 41% C 30% D 11% F 8% A 10% B 39% C 29% D 12% F 13% A 8% Number of Students Grade Distribution

Assessment and Curriculum Design  Two sides of the Same Coin

Design Course Goals  Course - goals  Modules - objectives  Activities - objectives

Develop Performance Expectations  Descriptions of Student Performance  Providing evidence convincing to you, your colleagues, and significant others that students have achieved a course goal or module objective  Descriptions of the conditions under which the evidence is collected

Assessment  Data collection with a purpose

What data are collected?  Measures of students’ knowledge and abilities –Answers to items on short answer tests –Essays –Position Papers –Oral Presentations –Poster Presentations

Assessments Linked to Goals  Goals articulated with outcomes that are measurable or observable (actions)

Functions of Assessments Functions of Assessments (Hodson 1992)  summative - description of students’ level of attainment  formative - diagnostic feedback to students and instructor  evaluative - provide instructors feedback about the effectiveness of the curriculum experiences  educative - engage students in interesting, challenging, significant experiences to develop further insight and understanding

Development of Rubric  Assessment tasks linked to outcomes  Define performance standard for biologically literate student.

Categories  Name of category  Criteria within categories -- discriminate among qualities you determine important  Reach consensus - students involved

Engagement problem  E.O. Wilson states that insects and other land- dwelling arthropods are so important that if all were to disappear, humanity probably could not last more than a few months. Based upon what you know today, do you believe that this statement is true? Write an essay and explain your reasoning as logically and thoroughly as you can. Do not use any outside resources for your essay.

What is the evidence we accept that students have learned?  What do we accept as a reasoned argument?  What are the criteria we use for evaluation of the argument?

WARRANT (W) EVIDENCE (E) CONCLUSION (C) Specific information relied on to support a given conclusion (E1, E2,...,En). Assertions put forth for acceptance. Statement justifying the move from Evidence to Conclusion “Evidence indicates that... E1, E2,...En.” “So... C” “Because E1, E2,...En, then C.” Argument Structure

Why use argument structure in science?  Consistent with nature of science  Justifies conclusions to the learning community  Develops problem solving and critical thinking skills  Develops art of questioning

What about content?  Argument and inquiry drive understanding of content  Content drives argument and inquiry

What did we find?  Students seldom disagreed with the statement  Some misconceptions about content appeared  Students rarely used warrants to support evidence

What did we do?

Revised Question  The keynote speaker at a scientific meeting made the statement that “insects and other land-dwelling arthropods are so important that if all were to disappear, humanity probably could not last more that a few months.” This generated considerable discussion, both pro and con at the meeting. What do you think about this statement? Construct an argument to explain your views and to illustrate your reasoning.

Question:  When we introduce innovation into our classes, how are student evaluations affected?

Self-Evaluation  Students who internalize valued achievement targets so thoroughly as to be able to confidently and completely evaluate their own and each other’s work, almost automatically become better performers in their own right.  McMillan and Forsyth (1991)

Goal 2 - Effectively describe biological concepts to peers  “So much group work made me realize that understanding a concept and being able to communicate a concept are different things.”  “...I find myself explaining things I was unable to explain before.”

Goal 3 - Confidence in ability to write about, criticize, analyze concepts in biology “I always was good at biology (or at least got good grades in biology), but never really understood any CONNECTIONS. This class made it clear to me that everything is connected to each other. We were forced to write down how we UNDERSTOOD concepts, not simply to memorize parts and functions.”

4 - Use process of scientific inquiry to think creatively and formulate questions... “...we did our position papers on two real world issues that we deal with every day. One paper we did was on the environment and the other was on breast cancer...This class really makes you think about how important scientists are in the world today. They try to formulated and answer questions that will help us survive in the future...”

7 - Positive interdependence; individual accountability within cooperative group  “Working in groups requires so much responsibility. This means coming to class every day, having your input for all of the questions and quizzes and most of all showing up on your own time to complete homework assignments, etc.... Working as a team requires dedication and cooperation This is how everyone will succeed.”

Factor 1Factor 2Factor 3 Change in Self-Efficacy + change no change - change Percent

So what?  Pre-service teacher education  Scientifically literate population  Research in the disciplines about learning  Recognizing, Evaluating, and Rewarding Excellent Undergraduate Teaching - NRC Study Committee on Higher Education