Assessment of JiTT on Student learning Kathleen A. Marrs, Ph.D. Department of Biology IUPUI, Indianapolis IN

Research Goals Question: Can a classroom environment using constructivist-based JiTT activities improve student performance in Biology? 1. Identify misconceptions students bring to Biology courses. 2. Create interactivity and active learning in large lecture class 3. Improve student study habits and class attendance 4. Increase relevance of studying Biology to everyday life 5. Enhance content knowledge in Biology Method: leverage technology to realize these goals! Outcome: Collect data to determine if this approach is effective in improving student study habits and content knowledge in Biology.

JiTT Improves Study Habits Warm Ups made a significant difference in student study skills. We asked students to report whether they ‘crammed’ for exams in Biology N100 (JiTT) vs. their other, non-JiTT classes: (n = 426) ‘Crammed’ in Biology N100 ‘Crammed’ in other courses A students 16%44% B students 34%63% C students 41%65% D students 64%71% F students 68%69%

JiTT Improves Graduate Student Study Habits 1) Did you put off studying for Biotech 540 and as a result ‘cram’ for Biotech 540 tests (saving virtually all studying for the day/night before the test)? (34.4%) Yes 2) Do you ‘cram’ for other courses that you have this semester? (62.0%) Yes 3) Do you feel that the Warm Up assignments and web notes made you stay ‘caught up’ on class material? (96.5%) Yes 4) Do you feel that you are more, less, or equally well ‘caught up’ on your other graduate courses this semester? (17.6%) Equal, (19.8%) More, (62.6%) Less Biology 540 (Graduate Biotechnology), n=37

JiTT Improves Class Preparation 1) Do you read the web notes before class? 2) Do you do the readings from the text before class? 3) Do you do read the textbook or the class notes (if provided) before your other classes? (1) Yes (2) Yes(3) Yes A students: 78%65% 53% B students: C students: D students: F students: Biology N100,n=162

Effect of Cooperative Learning Exercises on Student Learning a) Do you think the Cooperative Learning exercises help you to learn the class material? Yes (67%) b) Do you prefer doing the Cooperative Learning exercises to listening to lecture for 75 minutes? Yes (72%) c)Any positive or negative things you would like to say about them? Q4 = Cooperative Learning is a good thing to have. No offense, Dr. Marrs, but..…if it wasn't for the Coorperative Learning, I'd never come here. Q4 = I like the cooperative learning exercises because they're the only thing that we do in the class to participate, and it gets boring just listening. I have to stay active. Q4 = I think the exercises help me to understand the material a little better. I think maybe we do too many, but it definitely keeps me from getting too comfortable and falling asleep in class when I'm tired. They also take some of the pressure off of those horrible tests. Q4 = I think that the Cooperative Learning exercises help me to learn the material. They are a mini review for me!! They definitely help break up the tedium of lecture. They are wonderful!! Plus I like the fact that they can kind of take attendance for those of us who show up to class everyday!

More measures assessing JiTT’s effectiveness in teaching and learning Increased retention rates: (lower DFW rates) Examples: (Biology N100) 29% DFW lowered to 21% DFW. Increased cognitive gains: Examples: Measurable gains in Force Concept Inventory, a standardized test in Physics, and on Pre-vs. Post class tests in JiTT Biology classes. Increased classroom interactivity: based on reports from trained classroom observers. Students prefer JiTT: to traditional lecture classes, based on course attitude surveys, anonymous end-of-course evaluations, and student focus groups. “The web page was great. I’ve never had a class run from the web before and it was really neat. All of the different categories (Warm up, help, notes) were really great and made the class fun and interesting and different.”

JiTT and Content Knowledge …but do N100 students ‘learn’ the material any better with this approach? Evidence from Pre- and Post tests, with and without JiTT: Pre-test (% correct) Warm UpCL ExercisePost test (% correct) Mitosis15% 18% - Yes - Yes 62% 78% Genetics12% 20% - Yes - Yes 48% 64% Natural selection 10% 14% - Yes - Yes 25% 56% Human Population Growth 15% 17% - Yes - Yes 65% 85%

Pre-vs-Post: Mitosis Pre-test: In human cells, which of the following happens during mitosis? the duplication of all 46 chromosomes (23%) the replication of DNA(20%) the pairing of homologous chromosomes(11%) the fragmentation and disappearance of the nuclear envelope (18%) all of the above(28%) Warm Up question: What kind of ‘work’ does a cell in your liver do during Interphase? What happens to the DNA of that cell during the S-phase of Interphase? What kind of ‘work’ is that same cell doing during mitosis? What happens to the DNA of a cell during mitosis? Cooperative Learning: Mitosis and Cell Replacement: Drawing cells in different phases of the cell cycle. Post Test: In human cells, which of the following happens during mitosis? the duplication of all 46 chromosomes ( 5%) the replication of DNA( 9%) the pairing of homologous chromosomes( 6%) the fragmentation and disappearance of the nuclear envelope (78%)* all of the above( 2%) *(62%) without Warm Ups and Cooperative Learning

Pre-vs.-Post: Genetics Pre-test: A couple has just found out that they are both carriers for cystic fibrosis, a recessive metabolic disorder (carriers = Cc). What are the chances that their child will have the disorder cystic fibrosis (cc)? a)0%(12%) b)25%, or 1/4(20%) c)50% or 2/4(28%) d)75% or 3/4(21%) e) 100% or 4/4(19%) Warm Up: A couple who are in good health have a baby and are shocked to find that he has cystic fibrosis, an inherited, recessive disease. How did the child inherit a life-threatening disease from healthy people? Why weren't the parents affected by cystic fibrosis? Can this couple give birth to a child without cystic fibrosis? Why or why not? Cooperative Learning: Genetics problems. Post-test: A couple has just found out that they are both carriers for cystic fibrosis, a recessive metabolic disorder (carriers = Cc). What are the chances that their child will have the disorder cystic fibrosis (cc)? a)0%( 8%) b)25%, or 1/4(64%) *(48%) without Warm Ups and Cooperative Learning c)50% or 2/4(14%) d)75% or 3/4( 6%) e) 100% or 4/4( 8%)

Pre-vs.-Post: Natural Selection Pre Test: According to evidence gathered by Charles Darwin, evolution and natural selection is based on the _________ found in populations of organisms. a)acquired characteristics (15%) b)weakest members (16%) c)strongest members (45%) d)normal variations (14%) e)similarities (10%) Warm Up: “What does the term Survival of the Fittest mean? How does the environment influence the survival of an organism?” Cooperative Learning: Graphing evolution of antibiotic resistance over time. Post Test :According to evidence gathered by Charles Darwin, evolution and natural selection is based on the _________ found in populations of organisms. a)acquired characteristics( 4%) b)weakest members(10%) c)strongest members (23%) d)normal variations(56%)* e)similarities( 6%) *(25%) without Warm Ups and Cooperative Learning

Summary: JiTT Improves Content Knowledge in Biology %Gain (%G) = Percent of class correct on post-test - % of class correct on pre-test Average normalized gain: = % correct on post-test - % correct on pre-test / 100-(%Gain possible) We used the results from a 20-question pre-class and post-class test, calculating the average normalized gain as described (Hake 2002, reviewed in Dancy and Beichner 2002). We found that students tended to show an average normalized gain of about 15% on test questions about concepts that were discussed in class, but not necessarily reinforced by any additional class activities, and showed an average normalized gain of about 21% on test questions that were reinforced by back of the book homework problems. However, students tended to show an average normalized gain of ~52% on test questions that were reinforced by either Warm Up questions or Cooperative Learning activities, and showed an average normalized gain of ~60% on test questions reinforced by both Warm Up questions or Cooperative Learning activities (Marrs 2005, manuscript submitted).