1. Diane Ebert-May Department of Plant Biology Michigan State University ebertmay@msu.edu http://first2.org Pathways to Scientific Teaching

3. Team at MSU Rett Weber - Plant Biology (postdoctoral researcher) Deb Linton - Plant Biology (postdoctoral researcher) Duncan Sibley - Geology Doug Luckie - Physiology Scott Harrison - Microbiology (graduate student) Tammy Long - Plant Biology Heejun Lim - Chemistry Education Rob Pennock - Philosophy Charles Ofria - Engineering Rich Lenski - Microbiolgy Janet Batzli - Plant Biology [U of Wisconsin]

4. The trouble with our times is that the future is not what it used to be. -Paul Valery, The Art of Poetry

5. Engage

6. Question 1 Scientific teaching involves active learning strategies to engage students in the process of science. Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly disagree

7. Question 2 Students learn science best by doing science. Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly disagree

8. Question 3 How important is it to use multiple kinds of data to assess student learning? Please respond on a scale of 0-100 in increments of 10:

9. Question 4 How often do you use multiple kinds of data to make instructional decisions? Please respond on a scale of 0-100 in increments of 10:

10. Question 5 In my department, excellence/scholarship in teaching is rewarded at a level comparable to excellence/scholarship in research. Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly disagree

11. Question 1 Scientific teaching involves active learning strategies to engage students in the process of science. Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly disagree

13. Question 2 Students learn science best by doing science. Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly disagree

19. Large Class Meeting

20. Question 3 How important is it to use multiple kinds of data to assess student learning? Please respond on a scale of 0-100 in increments of 10:

21. Question 4 How often do you use multiple kinds of data to make instructional decisions? Please respond on a scale of 0-100 in increments of 10:

22. Question 5 In my department, excellence/scholarship in teaching is rewarded at a level comparable to excellence/scholarship in research. Please respond on a scale of 1-5: 1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly disagree

23. Explore

24. Teaching and learning challenges Developing and critiquing learning objectives and assessments Aligning objectives with assessment Exploring tools for assessment - what is the evidence? Implementing active learning strategies in science courses and curricula Topics for Today

25. What’s up with Termites? 1. On a sheet of paper, draw two circles near each other on the center of the page. 2. Release termites onto paper. 3. Keep creatures safe. I shall collect them in their original habitat. 4. What do you observe about termite behavior? 5. Develop a question your group could explore if you had more time. (15 minutes - select a timekeeper)

26. 1. Develop one possible objective for this ‘inquiry’. Team Written response. Reporter - Recorder - Timekeeper - (10 minutes) Checker - 2. Develop an assessment appropriate for the objective.

27. Explore...thin air

28. What is going on? Teaching without learning! Talk to your neighbor - brainstorm 1. Diagnose situation - the learning challenge 2. Where is/are the missing links?misconceptions?

29. In your groups: What do you want your students to be able to DO? Design learning objectives that address photosynthesis learning challenges. Get to know your students and their prior knowledge. May use pretest or diagnostic (clicker) question

30. At the end of this unit, students should be able to…. Generate statement of measurable performance

31. Learning objectives Once you have a set of learning goals…. assign a Bloom-level to each goal. Statement that indicates level of expectation of performance

32. What level of learning do we ask of our students? Bloom (1956) Cognitive Domain of Educational Objectives 6 categories - Knowledge Comprehension Application Analysis Synthesis Evaluation

33. Learning Objectives... Revise

34. Explore more...

36. Jigsaw New groups Count off -- 1, 2, 3, 4, 5 All 1s work on same paper....2s, 3s, 4s, 5s Return to ‘home’ groups and share what you found in each of the papers. Report out

37. Paper Assignments Group 1: Climate change.... Group 2: Novel assessments... Group 3: Practicing scientific inquiry... Group 4: Determining confidence... Group 5: Collaborative learning...

38. In your groups: Read the paper, discuss, record... 1. What are the student learning goals? 2. What is the Bloom-level of each goal? 3. Do the assessments align with the goals? 4. What are the active learning strategies?

39. What is assessment? Data collection with the purpose of answering questions about… students’ understanding students’ attitudes students’ skills instructional design and implementation curricular reform (at multiple grainsizes)

40. Why do assessment? 1. Improve student learning and development. 2. Provide students and faculty substantive feedback about student understanding. 3. Challenge to use disciplinary research strategies to assess learning.

41. Multiple Choice … … Concept Maps … … Essay … … Interview high Ease of Assessment low low Potential for Assessment of Learning high Theoretical Framework Ausubel 1968; meaningful learning Novak 1998; visual representations King and Kitchner 1994; reflective judgment National Research Council 1999; theoretical frameworks for assessment Assessment Gradient

42. Identify desired goals/objectives Determine acceptable evidence Design learning experiences and instruction Wiggins and McTighe 1998 Backwards Design

43. Department of Plant Biology Course Level Mean Bloom Level

44. Department of Plant Biology % Total Qs Course Level *N items Bloom Level

45. Explain

46. Assessments: Tools to Detect Learning

47. What’s a tool used to test student attitude? 2.VASS (Views) 2 2 1. Self-Efficacy

48. What’s a tool used to test knowledge? 2 2 1. Exam2. Writing3. PBA4. Concept I

49. The most common tool used is?

50. What is the relationship among DNA, a gene, and a chromosome? a. A chromosome contains hundreds of genes which are composed of DNA. b. DNA contains hundreds of genes which are composed of chromosomes. c. A gene contains hundreds of chromosomes which are composed of DNA.

51. An mRNA molecule with the following sequence is being read by a ribosome: 5' CCG-ACG -UGC 3' What was the original 'antisense' DNA sequence that encoded this? a. 5' CCG-ACG-UGC 3' b. 5' CCG-ACG-TGC 3' c. 5' GGC-TGC-ACG 3' d. 5' CCA-CGT-CGG 3'

52. What is the relationship among DNA, a gene, and a chromosome? a. A chromosome contains hundreds of genes which are composed of DNA. b. DNA contains hundreds of genes which are composed of chromosomes. c. A gene contains hundreds of chromosomes which are composed of DNA.

53. What is the relationship among DNA, a gene, and a chromosome? a. A chromosome contains hundreds of genes which are composed of DNA. b. DNA contains hundreds of genes which are composed of chromosomes. c. A gene contains hundreds of chromosomes which are composed of DNA. What is the relationship among DNA, a gene, and a chromosome? a. A chromosome contains hundreds of genes which are composed of DNA. b. DNA contains hundreds of genes which are composed of chromosomes. c. A gene contains hundreds of chromosomes which are composed of DNA.

54. An mRNA molecule with the following sequence is being read by a ribosome: 5' CCG-ACG -UGC 3‘ What was the original 'antisense' DNA sequence that encoded this? a. 5' CCG-ACG-UGC 3‘ b. 5' CCG-ACG-TGC 3‘ c. 5' GGC-TGC-ACG 3‘ d. 5' CCA-CGT-CGG 3'

55. An mRNA molecule with the following sequence is being read by a ribosome: 5' CCG-ACG -UGC 3‘ What was the original 'antisense' DNA sequence that encoded this? a. 5' CCG-ACG-UGC 3‘ b. 5' CCG-ACG-TGC 3‘ c. 5' GGC-TGC-ACG 3‘ d. 5' CCA-CGT-CGG 3'

56. An mRNA molecule with the following sequence is being read by a ribosome: 5' CCG-ACG -UGC 3' What was the original 'antisense' DNA sequence that encoded this? a. 5' CCG-ACG-UGC 3‘ b. 5' CCG-ACG-TGC 3‘ c. 5' GGC-TGC-ACG 3‘ d. 3' GGC-TGC-ACG 5'

57. Problem Identification

58. DEFINE a bicycle with words DRAW a picture of a bicycle Let’s try an experiment OK?

59. Which parts and subsystems did you include in your drawing vs. your definition? Now... let’s examine the concept of a bicycle

60. What’s a subsystem? Propulsion

61. 61 Joseph Novak, in Learning, Creating and Using Knowledge: Concept Maps as Facilitative Tools in Schools and Corporations 1998 “Meaningful learning results when the learner chooses to relate new information to ideas the learner already knows. Rote learning occurs when the learner memorizes new information without relating to prior knowledge.”

62. C-TOOLS Students could explain details of transcription & translation but not the relation... “DNA-Gene- Chromosome” Can concept mapping force students to “Think different” and confront their (mis) understanding?

63. Hierarchy has Structure has Concept Maps Visual Diagrams Or Models are represent Knowledge or Understanding Concepts display connected with Linking Words Used for AssessmentOrganization Reflection & Learning promotes Context is constructed with New Information Prior Knowledge

64. We’ve built some infrastructure... With Robograder for feedback.

65. www.ctools.msu.edu

66. Student’s Concept Map

67. “In a pair of mutually complementary studies*, concept mapping has been shown to facilitate learning when constructed both before a unit of instruction, as an advance organizer, or when constructed following the unit, as a so-called ‘postorganizer.’ ” *Willerman and MacHarg (1991), Spiegel and Barufaldi (1994) D.E. Moody, Mapping Biology Knowledge, 2000 Concept Maps have been tested in the classroom.

68. 1. Select a concept that is critical for your students to understand. 2. Identify 4 or 5 subconcepts that are important to understanding that concept e.g., DNA - Gene- Chromosome - Enzyme For a course you teach..... 3. Arrange them by rank order - top most general, bottom most specific 4. Add linking lines to make connections between two concepts 5. Add linking words that describe the relationship between two concepts

69. Objective (outcome) Students will demonstrate understanding of evolution by natural selection.

70. Changes in a population occur through a gradual change in individual members of a population. New traits in species are developed in response to need. All members of a population are genetically equivalent, variation and fitness are not considered. Traits acquired during an individual’s lifetime will be inherited by offspring. Alternative Conceptions: Natural Selection

71. (AAAS 1999) Pre-test: Essay Explain the changes that occurred in the tree and animal. Use your current understanding of evolution by natural selection.

72. Modified problem: populations of trees and animals

73. How do we develop rubrics? Describe the goal/objective for the activity, problem, task... Select the assessment tasks aligned with goals Develop performance standards Differentiate levels of responses based on clearly described criteria Rate (assign value) the categories

74. Scoring Rubric for Quizzes and Homework

75. Advantages of Scoring Rubrics Improve the reliability of scoring written assignments and oral presentations Convey goals and performance expectations of students in an unambiguous way Convey “grading standards” or “point values” and relate them to performance goals Engage students in critical evaluation of their own performance Save time but spend it well

76. Limitations of Scoring Rubrics Problem of criteria Problem of practice and regular use Scoring Rubric website http://www.wcer.wisc.edu/nise/cl1/flag/ Sample Rubrics for Environmental Science http://www.msu.edu/~ebertmay/isb202/home.html

77. Guidelines for Planning Research How do instructors move from assessment to designing research on learning?

78. What did the assessment data suggest about student understanding? Why didn’t students understand critical concepts? What has been done already about students’ understanding of these concepts? Ask Questions

79. Investigator’s Purpose Improve own classroom instruction? Really conduct research?

80. How and why will you select the research methods? What kinds of data will you collect? Direct or Self-Report? How will you analyze the data? Design Study Collect Data

81. Research Designs

82. Data Collection

83. How will you analyze the data? How could the results influence instruction? Analyze Data

84. Quantitative data - statistical analysis Qualitative data break into manageable units and define coding categories search for patterns, quantify interpret and synthesize Valid and repeatable measures Data Analysis

85. Ideas and results are peer reviewed - formally and/or informally. Science journals Report the Study

86. Articles derived from journal papers

87. How do analogous assessment questions help us determine students’ prior understanding and progressive thinking about the carbon cycle? Question

88. Concept 1: Matter disappears during decomposition of organisms in the soil. Concept 2: Photosynthesis as Energy: Photosynthesis provides energy for uptake of nutrients through roots which builds biomass. No biomass built through photosynthesis alone. Concept 3: Thin Air: CO 2 and O 2 are gases therefore, do not have mass and therefore, can not add or take away mass from an organism. Concept 4: Plant Altruism: CO 2 is converted to O 2 in plant leaves so that all organisms can ‘breathe’. Concept 5: All Green: Plants have chloroplasts instead of mitochondria so they can not respire. Some Common Misconceptions about Photosynthesis & Respiration Some Common Misconceptions about Photosynthesis & Respiration

89. Ebert-May et al. 2003 Bioscience Design Experiment Quantitative Data Qualitative Data

90. Instructional Design Two class meetings on carbon cycle (160 minutes) Active, inquiry-based learning Cooperative groups Questions, group processing, large lecture sections, small discussion sections, multi-week laboratory investigation Homework problems including web-based modules Different faculty for each course One graduate/8-10 undergraduate TAs per course

91. Experimental Design Two introductory courses for majors: Bio 1 - organismal/population biology (faculty A) Bio 2 - cell and molecular biology (faculty B) Three cohorts: Cohort 1Bio 1 (n=141) Cohort 2Bio1/Bio2 (n=63) Cohort 3 Other/Bio2 (n=40)

92. Assessment Design Multiple iterations/versions of the carbon cycle problem Pretest, midterm, final with additional formative assessments during class Administered during instruction Semester 1 - pretest, midterm, final exam Semester 2 - final exam Text

93. Problem Experimental setup: Weighed out 3 batches of radish seeds each weighing 1.5 g. Experimental treatments: 1. Seeds placed on moistened paper towels in LIGHT 2. Seeds placed on moistened paper towels in DARK 3. Seeds not moistened (left DRY) placed in light

94. Problem (2) After 1 week, all plant material was dried in an oven overnight (no water left) and plant biomass was measured in grams. Predict the biomass of the plant material in the various treatments. Water, light Water, dark No water, light

95. Results Mass of Radish Seeds/Seedlings 1.46 g 1.63 g 1.20 g Write an explanation about the results. Explain the results. Write individually on carbonless paper.

96. Grandma Johnson Problem Hypothetical scenario: Grandma Johnson had very sentimental feelings toward Johnson Canyon, Utah, where she and her late husband had honeymooned long ago. Her feelings toward this spot were such that upon her death she requested to be buried under a creosote bush overlooking the canyon. Trace the path of a carbon atom from Grandma Johnson’s remains to where it could become part of a coyote. NOTE: the coyote will not dig up Grandma Johnson and consume any of her remains.

97. Analysis of Responses Used same scoring rubric (coding scheme) for all three problems - calibrated by adding additional criteria when necessary, rescoring: Examined two major concepts: Concept 1: Decomposers respire CO 2 Concept 2: Plants uptake of CO 2 Explanations categorized into two groups: Organisms (trophic levels) Processes (metabolic)

98. Coding Scheme

99. Correct Student Responses (%) Cellular Respiration by Decomposers Bio1/Bio2Other/Bio2 Friedmans, p<0.01 Cellular Respiration by Decomposers

100. Pathway of Carbon in Photosynthesis Bio1/Bio2 Correct Student Responses (%) Other/Bio2 Friedmans, p<0.05 Pathway of Carbon in Photosynthesis

101. Evolution and Natural Selection How does active learning affect students’ understanding of evolution and natural selection over time?

102. In guppy populations, what are the primary changes that occur gradually over time? a. The traits of each individual guppy within a population gradually change. b. The proportions of guppies having different traits within a population change. c. Successful behaviors learned by certain guppies are passed on to offspring. d. Mutations occur to meet the needs of the guppies as the environment changes. CINS Multiple Choice

103. Day 13 in class: Pretest: CINS multiple choice and dino essay Artificial selection in dogs Groups discuss and record: Why are dogs only one species? Groups discuss and record: Breed a larger dog. Assign concept map 4 - homework Active Learning

104. Individual Assignment Make a new concept map demonstrating your understanding of natural selection using the following concepts. genetic variation evolution species population natural selection artificial selection selective agent Concept Map 4

105. Days of Instruction Mean Percent Correct MC Essay C-map Clicker Assessments Pre MC Pre Essay Active Learning

106. Day 14 in class Artificial selection in plants Group discuss and record: What traits were selected and why? Clicker Q: Plant fitness Natural selection Clicker Q: Selection on beak size Group discuss and record: What traits were naturally selected in Hawaiian finches? Minute paper: Natural and artificial selection Assign homework: Guppy sexual selection Active Learning

107. Days of Instruction Mean Percent Correct MC Essay Assessments Pre MC Pre Essay Clicker MC Active Learning

108. http://www.pbs.org/wgbh/evolution/sex/guppy/ed_pop.html Homework: Guppy Sexual Selection

109. Days of Instruction Mean Percent Correct MC Essay Assessments Pre MC Pre Essay Clicker MC Active Learning C-map 4

110. Day 15 In class: groups Determine fitness of wild tobacco Sexual selection review guppy homework Assign homework: concept map 5 Active Learning

111. Days of Instruction Mean Percent Correct C-map 5 Assessments C-map 4 Pre MC Pre Essay Clicker MC Active Learning

112. Individual Assignment Add the following terms to concept map 4. biotic factors abiotic factors fitness trait mutation reproduction meiosis allele frequency variation selection pressure adaptation Concept Map 5

113. 2nd midterm exam CINS multiple choice on natural selection Final exam CINS multiple choice on natural selection Dinosaur post test Assessments

114. Days of Instruction Mean Percent Correct Pre MC Post 1 MC Post 2 MC Post Essay Pre Essay Clicker MC Active Learning Assessments 2005 C-map 5 C-map 4 day Correlations (p<0.05) N= 49-117 Pre MC - Pre Essay Pre MC - Post 1 MC Pre MC - Post 2 MC Pre Essay - Post 1 MC Pre Essay - Post Essay 0.35 0.54 0.25 0.33 0.53 Cmap 4 - Cmap 5 Cmap 4 - Post 2 MC Post 1 MC - Post 2 MC 0.60 0.38 0.29

115. Days of Instruction Mean Percent Correct Post 1 MC Active Learning C-map Assessments 2004 Post 2 MC Post 1 Essay Pre Essay Clicker 1 MC Clicker 2 MC Correlations (p<0.05) N=84-103 Post 2 Essay Pre Essay - Post 1 Essay Clicker 2 MC - Post 1 MC Post 1 MC - Post 2 MC Post 1 MC - Post 2 Essay 0.25 0.46 0.42 Post 1 Essay - Post 2 MC Post 1 Essay - Post 2 Essay Post 2 MC - Post 2 Essay 0.24 0.29 0.60

116. System Model

117. ResearchTeaching Is it scientific?