1. DIAGNOSTIC QUESTION CLUSTER (DQC) PROJECT Project goals, overview Concept Inventories (FCI) & Misconceptions DCQs – what they are, how developed, pedagogical framework Faculty development component – using DQCs and active learning to change biology teaching and learning Some findings for faculty we’ve been working with John Merrill, Kathy Williams & Tod Duncan – their experiences and suggestions

2. DIAGNOSTIC QUESTION CLUSTER PROJECT Charlene D’Avanzo, Andy Anderson, John Merrill, Kathy Williams, Nancy Pelaez, Alan Griffith, Laurel Hartley, Brook Wilke, Nancy Stamp … FOCUSES ON “BIOLOGICAL REASONING” An observation: When faced with a question or observation - most Introductory Biology level students don’t use the fundamental principles and reasoning that biologists use. Most biology faculty do not teach students how to use fundamental principles and reasoning used by practicing biologists to address questions.

3. Aspects of DQC Project Centers on “clusters” (~4-6) of research-based questions designed to examine students’ reasoning and understanding about energy and matter – subcellular-ecosystem Questions are used pre/post (module, course) Questions are intended to be used with active teaching that targets problematic reasoning Very intentionally integrates faculty development & modification and use of the DQCs – e.g. faculty contribute to the research and help modify the questions

4. THIS IS VERY MUCH A PROJECT IN PROCESS – many questions …

5. A little background The FCI and Biology Misconceptions

6. FORCE CONCEPT INVENTORY – FCI 30 multiple choice questions given as pre/post tests – Newtonian mechanics – for experts the answers are very obvious (D. Hestenes 1992)

7. Eric Mazur - How could these undoubtedly bright students, capable of solving complicated problems, fail on these simple questions? Pre/post testing; focus on a few very key concepts; ‘clicker questions’; listen to the students and talk with the students

8. “Minds of Our Own” – Private Universe series Harvard graduates are given a piece of wood and a seed and asked: “This seed can grow into a big tree. Where does the mass in the tree come from?” The great majority will say “from the soil”.

9. HOWEVER, FOCUSING ON “MISCONCEPTIONS” IS LIMITING Too many misconceptions: Our lists get so long that they aren’t useful any more Too idiosyncratic - don’t tell us enough about general problematic patterns in students’ thinking about biology How do we help faculty focus more on conceptual thinking?

10. THE DQCs – Research, Development, Refinement Conceptual Framework

11. Andy Anderson and colleagues at Michigan State University What Anderson et al. did to develop the DQCs – 1. Talked to lots of students about their reasoning & understanding when presented with biology questions and problems. Tried to categorize students’ reasoning by the key concepts or principles they are not using. 2. Developed diagnostic questions from the students responses – e.g. the ‘distracters’ in multiple choice questions come from students’ responses and explanations

12. DQC DEVELOPMENT & REFINEMENT MILLION DOLLAR QUESTIONS

13. FRAMEWORK for the DQCs PROCESSES – Key processes of metabolism and C cycling - Generation (photosynthesis, plant growth, primary production); - Transformation (digestion, biosynthesis, plant/animal growth, food webs) ; Oxidation (cellular respiration, animal movement, weight loss, combustion, energy pyramids) PRINCIPLES - “ Hidden curriculum” - c onstraints on all processes that are taken for granted by professors, but not understood or applied by students – Conservation of Matter and Energy SCALES – Processes transform matter and energy and change systems at multiple scales: atomic-molecular, microscopic-cellular, macroscopic-organismal, Large-scale/ecosystem/global

14. PRINCIPLED REASONING – Biologists understand that there are fundamental principles that constrain biological phenomena – e.g. Conservation of Matter and Conservation of Energy. Atoms don’t disappear and reappear – ‘keep track’ of them Energy does not just ‘turn into’ matter and v.v. (biological sense) Same principles are applied across all scales of organization - subcellular, cellular, organismal and ecosystem

15. DQC WEBSITE – biodqc.org

16. Summary - The DQCs Application questions assess student understanding of foundational biological principles in different contexts Multiple choice, true-false, open-response – can be completed in 15-20 min Were not designed as test questions (but are being used in that way) Pre/post Used with active teaching Focus on ‘Principled Reasoning’ Faculty are ‘trained’ in workshops to score open responses, how to use the questions, etc

17. Example Question: Process: Photosynthesis Principles – Energy Matter Level - organismal

18. FACULTY DEVELOPMENT QUESTIONS When provided with research-based DQCs – what would faculty “do”? Does use of the DQCs, active learning, the Framework, etc. change faculty thinking about teaching & learning? How long does that take? What’s important? Differences between institutions?

19. THE FACULTY WE WORK WITH From a wide range of institutions nationally (15 per group) – universities, 4 year colleges, community colleges Selected via an application process Attend several workshops (annual) Work in teams; lots of interaction during the year (conference calls etc.) First group was ecologists; second group is wide range of faculty teaching introductory biology Faculty from Group 1 working with Group 2

20. IDEALLY AFTER OUR WORKSHOP – FACULTY WOULD Select a set of DQCs appropriate for their course/section of course Give students the pre-test – maybe before a module (e.g. on Photosynthesis) Look at the data, score the written responses – see the problematic patterns Focus their teaching on principles students don’t understand; show students the data and discuss it with them. Ask why, what they think Use targeted active teaching strategies (e.g. on our website) Give the post-test to assess progress

21. FINDINGS WITH FIRST GROUP OF FACULTY Formative use of DQCs as diagnostic information: in year one although all participants gave the specific DQCs as pre/post tests, nearly all used them as summative information – not to inform instruction during the course. Then, they were surprised by the poor performance of THEIR students and this motivated them to begin to change their teaching next time For some, association of the questions and the framework has been fuzzy Confounded, though (our learning how to do this work with faculty) -But, it seems that several cycles are needed before faculty to start to change their thinking in fundamental ways. Change is necessarily incremental. -They also need a good deal of support both from us and from their teammates -They need to have the time, opportunity, tools and support. This is very hard for some faculty (community college, R-1)

22. Fundamental change is very hard and very slow

23. SOME FINDINGS Basic ecology courses – 495 students in 12 institutions, 2008-9 More than 70% of students did not use principled reasoning even in post tests No difference across institutions Students displayed all the expected problems

24. Importance of – The SCHOLARSHIP OF TEACHING AND LEARNING

25. Update We are working with our second set of 15 faculty – one workshop in the fall for spring courses One team – ‘Biology Directors’ – are BLCers Ruth Buskirk, Jo Kurdziel, Melissa Michael, Jean Heitz, John Merrill

26. ACTIVE TEACHING EXAMPLES in class, homework - DQC as a clicker question - Process tool - Measurements of CO 2, temp.

28. John Merrill & Kathy Williams – how they use the DQCs in their Intro Bio courses Tod Duncan – an active learning example

29. “ Mice in a box”