1. Improving scientific thinking and experimental design skills in undergraduate students Angela Hodgson, North Dakota State University Fiona Rawle, University of Toronto at Mississauga.

2. How did this collaboration start?
Talked at BLC
ed each other after BLC
Applied for and received catalytic grant
THANK YOU Jane Reece, the family of Neil Campbell, and Pearson for your support!

3. Rationale – Science Process Skills
What are science process skills?
Experimental Design
Graphing and visualizing data
Statistical analysis and hypothesis testing
Scientific communication
Why are science process skills important?
-we both started thinking about science process skills separately. As a result, we came up with different list of learning goals

4. Comparison of Classes UTM NDSU Class Intro to Evolution
General Biology I
Size
2 x 380
2 x
Lab Size
24; weekly 1.5 hr lab
24; weekly 2 hr lab
Semester Length
12 weeks
16 weeks

5. Learning Outcomes UTM intervention NDSU Intervention
Explain what a hypothesis is.
Identify good vs bad hypotheses.
Identify elements of good experimental design.
Identify independent and dependent variables in an experimental design.
Explain the importance of replication in experimental design.
Critique an experiment.
Improve upon a prior experimental design.
Design an experiment to test a hypothesis.
Write a good hypothesis
Identify independent and dependent variables in an experimental design
Design an experiment to test a hypothesis.
Record and analyze scientific data in a spreadsheet
Perform a statistical hypothesis test
Create graph that correctly represents scientific data
Interpret and synthesize results from a scientific experiment
Modeling good backward design.

6. Introductory Biology Course Re-Design
1. “Thinking Like a Scientist” Introductory Module
2. Case Study Based Tutorial Sessions
3. Inquiry Based Labs
4. “Science vs. Pseudoscience” Lecture Examples
5. Scientific Literacy Assignment
6. Active Learning Exercises in Lecture
UTM – Course Design

7. Scientific Literature
Lecture Intervention
Lab Intervention
Assessment 1.
EDAT
Module:
Thinking Like
a Scientist
Worksheets on
hypothesis
Testing and
experimental
design included
in every lab
Module:
Experimental
Design
Module: Critiquing
Scientific Literature
Group
Experimental
Design
Brainstorming
Session
Group EDAT
Science Process
Skills Concept
Assessment
Science vs
Pseudoscience
Clicker Case Studies
12.
EDAT, Final Exam

8. In class example - UTM
“You should take chlorophyll supplements because your need to oxygenate your bowel”
(See “Bad Science” by Ben Goldacre)

9. Inquiry Lab – Optimization of ethanol production (Yeast respiration)
Week # 1.
16.
Lab Intervention
Assessment
EDAT pretest
EDAT, SPARST
2nd Lab Practical
Inquiry Lab – Optimization of
ethanol production
(Yeast respiration)
Inquiry Lab: Optimization of
commercial osmotic dehydration
(Diffusion and osmosis)
1st Lab Practical
Inquiry Lab: Optimize greenhouse
photosynthetic rate
(Spectrophotometry and Photosynthesis
Inquiry Lab: Does race exist?
From Peggy Brickman
(ABLE proceedings Vol. 32)
(DNA electrophoresis)

10. NDSU – Example Inquiry Lab
Question
Agri-gas is a company currently producing ethanol by yeast fermentation of a 5% sucrose solution at 20°C. The company would like to increase the metabolic rate of the yeast in order to maximize ethanol production. Would changing any aspect of the current production system increase the rate of yeast fermentation?
Materials
Incubators that can be set at 40 degrees C and 60 degrees C, refrigerator (4 degrees C), carbohydrates- sucrose(table sugar), glucose, brown sugar, powdered sugar, lamps with different color light bulbs, dark cabinets, HCl (acid), base (NaOH)

11. Experimental Design Aptitude Test (Sirum, 2011 Bioscience)
Students taking the EDAT are given the following prompt:
Advertisements for a herbal product, ginseng, claim that it promotes endurance. To determine if the claim is fraudulent and prior to accepting this claim, what type of evidence would you like to see? Provide details of an investigative design.

12. EDAT Scoring Rubric
1 pt. is awarded for each of the following rubric items that are included in the student answer.
Recognition that an experiment can be done to test the claim
Identification of what variable is manipulated
Identification of what variable is measured
Description of how dependent variable is measured
Realization that there is one other variable that must be held constant
Understanding of the placebo effect
Realization that there are many variables that must be held constant
Understanding that the larger the sample size or number of subjects, the better the data.
Understanding that the experiment needs to be repeated.
Awareness that one can never prove a hypothesis, that one can never be 100% sure.

13. UTM: Pre and Post EDAT

14. UTM: Pre and Post Rubric Item Score

15. NDSU: Pre and Post EDAT
N=517

16. NDSU: Pre and Post Rubric Item Score

17. Learning Outcomes Write a good hypothesis
Identify independent and dependent variables in an experimental design
Design an experiment to test a hypothesis.
Record and analyze scientific data in a spreadsheet
Perform a statistical hypothesis test
Create graph that correctly represents scientific data
Interpret and synthesize results from a scientific experiment
Modeling good backward design.

18. NDSU: Additional Rubric Items
11. Identified that a hypothesis is needed/wrote a hypothesis.
12. Identified that a null hypothesis is needed for hypothesis testing, or wrote a null hypothesis.
13. Stated that treatment means need to be significantly different to reject the null hypothesis.

19. NDSU: Additional Pre and Post Rubric Item Scores
Analysis of the results above showed that NDSU students made good gains on these three rubric items that more closely matched the NDSU learning objectives. 

20. What we learned?

21. Future Directions
Combine the NDSU (inquiry labs) and UTM (lecture modules) interventions
Use an expanded rubric for assessment.
Use the Science Process and Reasoning Skills Test (SPARST) developed by Clarissa Dirks and Mary Pat Wenderoth.
Mention rubric developed by Clarissa and Mary Pat.
-EDAT is a good starting point but it doesn’t give us the level of differentiation we want between our students. For example – is there a hypothesis? Testable hypothesis? Good control?

22. Acknowledgements
This project was funded by a Catalytic Grant. We thank Jane Reece, the family of Neil Campbell, and Pearson for this opportunity.
Clarissa Dirks and Mary Pat Wenderoth gave us access to SPARST, which we also used as a post-test.
We also thank our students that took part in this study, and our teaching assistants.

23. Science Process and Reasoning Skills Test
Learning Outcome
Identify a hypothesis
Identify a controlling variable
Evaluate the quality of the experiment to test the hypothesis
Choose a supporting hypothesis
Analyze treatments and controlling for variables
Apply knowledge of appropriate controls
Select prediction
Evaluate experimental design to determine if the conclusion is warranted
Evaluate experimental designs to determine which will yield valid results
Compare and identify the variables for which one should control