1. CREATE Mini Conference Melanie M Cooper Michigan State University May 2013

2. Todays students  Will have on average > 8 jobs  May have jobs that have not been invented yet  (~ 65% of them according to NYT)  Need skills that are not necessarily emphasized in college

3. The time is right They could drive significant improvements in STEM education. Proposed changes to both K-12 and college STEM education are upon us

5. BUT, K-12 and higher education have a chicken and egg problem

6. Students arrive at college underprepared College faculty blame K-12 system But teachers learn content in our STEM courses Methods classes don’t help if not integrated with appropriate content Teachers teach what and how they were taught STEM content There is a huge need for STEM disciplinary departments to engage in education research and practice

7. PCAST agrees!

8. PCAST Recommendations 1.Catalyze widespread adoption of empirically validated teaching practices. 2.Advocate and provide support for replacing standard laboratory courses with discovery based research courses. 3.Launch a national experiment in postsecondary mathematics education to address the math preparation gap. 4.Encourage partnerships among stakeholders to diversify pathways to STEM careers. 5.Create a Presidential Council on STEM Education http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-engage- to-excel-final_feb.pdf

9. PCAST Recommendations 1.Catalyze widespread adoption of empirically validated teaching practices. 2.Advocate and provide support for replacing standard laboratory courses with discovery based research courses. 3.Launch a national experiment in postsecondary mathematics education to address the math preparation gap. 4.Encourage partnerships among stakeholders to diversify pathways to STEM careers. 5.Create a Presidential Council on STEM Education http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-engage- to-excel-final_feb.pdf

10. PCAST Recommendation 1 1.1Establish discipline-focused programs funded by Federal research agencies, academic institutions, disciplinary societies, and foundations to train current and future faculty in evidence- based teaching practices. 1.2Create a “STEM Institutional Transformation Awards” competitive grants program at NSF 1.3Request that the National Academies develop metrics to evaluate STEM education

11. 1.1 Evidence-Based Teaching Practices?

12. Discipline Based Education Research “DBER combines knowledge of teaching and learning with deep knowledge of discipline based STEM content. It describes the specific difficulties learners face and the specialized intellectual and instructional resources that can facilitate student understanding” www.nap.edu/catalog.php?record_id=13362

13. Can Impact How we teachHow students learn What we teachThe curriculum Who we teach Issues of access and diversity How we know students have learned Assessment DBER

14. Focus on College level Discipline Specific Used by College Faculty Science Education Focus on K-12 Used by trained teachers Cognitive/Educational Psychology Focus on learning Not discipline specific SoTL Focus on College level Often inter- disciplinary But really these categories are fuzzy – lots of overlap How is DBER different than other areas of research in teaching and learning?

15. The NRC report will provide focus and future directions for the growing field of DBER

16. To meet the challenges involved in improving STEM education, collaboration is the KEY

17. 1.2Create a “STEM Institutional Transformation Awards” competitive grants program at NSF

18. 1.3Request that the National Academies develop metrics to evaluate STEM education This is where the rubber hits the road!

19. “to educate and improve student performance, not merely to audit it” Assessment: Wiggins, G. (1998). Educative assessment: Designing assessments to inform and improve student performance. San Francisco, CA: Jossey-Bass “if you don’t assess what’s important, what’s assessed becomes important”

20. Our ultimate goal: to allow students to learn the knowledge and skills that can be applied to new situations Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century, NRC 2012

21. How to assess/evaluate what we say we value is a major issue

22. A cautionary tale Year%DFWACS %ile# students# DFW 200123611200270 200230721199362 200335721314453 200444751429625 200523721265290 200619721260240 200711761306150 200818791300230 200915751570236

23. Reforms  Weekly meetings to negotiate “big ideas” and learning outcomes and assessments (backward design)  Reduce class size (to about 100 from 180)  Remove content (~30%)  Add “active” learning (group work, clickers etc)  Each faculty member uses their own notes/class management style  there were typically no difference in grade distribution regardless of amount of “active learning”

24. Success! Year%DFWACS %ile# students# DFW 200123611200270 200230721199362 200335721314453 200444751429625 200523721265290 200619721260240 200711761306150 200818791300230 200915751570236 We can go home

25. Not so fast… What happens to the potential energy when you bring two hydrogen atoms together? “The potential energy goes up … when you break a bond it releases energy”

26. Energy changes and bonding % Students by type with bond energy misconceptions Chemistry level (#)% General Chemistry (77)50 Inorganic (13)54 Organic (172)65 Analytical (35)51 Physical (16)56 Graduate Students (21)68 Post docs (80)68

27. In our studies we found that good students (hard working, conscientious, bright A students) had significant problems understanding core disciplinary ideas, and were unable to apply their knowledge to new situations

28. NRC Report: Assessing 21 st century skills

29. 21 st Century Skills  Cognitive skills: nonroutine problem solving, critical thinking, systems thinking  Interpersonal skills: complex communication, social skills, teamwork, cultural sensitivity, dealing with diversity  Intrapersonal skills: self-management, time management, self development, self-regulation, adaptability, executive functioning

30. NRC report: Assessing 21 st Century Skills (2011) Economy-wide measures of routine and nonroutine task input: 1969-1998 (1969 = 0). SOURCE: Levi and Murnane (2004).

31. The Next Generation Science Standards (NGSS) provide us with a potential model to drive development of new assessments

32. Scientific and Engineering Practices 1. Asking questions and defining problems 2. Developing and using models 3. Planning and carrying out investigations and design solutions 4. Analyzing and interpreting data 5. Using mathematics and computational thinking 6. Developing explanations and designing solutions 7. Engaging in argument from evidence 8. Obtaining, evaluating, and communicating information The multiple ways of knowing and doing that scientists and engineers use to study the natural world and design world.

33. Performance Expectation Cross Cutting Science Practices Core Content NGSS Performance Expectations drive assesment

34. HS-PS1-4 “Develop a model to illustrate that the release or absorption of energy from a chemical reaction system depends upon the changes in total bond energy.” Performance Expectations

35. HS-PS-1c. Plan and conduct an investigation to gather evidence to compare the structure of substances at the bulk scale to infer the electrical forces within and between atoms.

36. How many molecules are in 1.00 kg of hydrazine, N 2 H 4 ? A.9.40  10 21 B.1.88  10 25 C.1.88  10 22 D.1.13  10 26 E.1.13  10 23 Not this

37. But This

38. Use technology to design, administer, grade, model and evaluate student performances

39. Problem Solving Strategies -captured during student performance Stevens, R., Soller, A., Cooper, M., Sprang, M., (2004) Computer Science Editorial III, Springer- Verlag, Heidelberg Germany

40. Assessment of Problem Solving IMMEX strategy (ANN HMM) (concurrent) IMMEX ability (IRT) (concurrent) Metacognitive Activity(MCAi) (pre/post Cooper, M. M.; Sandi-Urena, S., Chem. Educ. Res. Pract., 2008, 9, 18-24 Across method and time.

41. beSocratic  Allows, recognizes and responds to free form input – graphs, chemical structures, diagrams, gestures, and some text  Provides tiered, contextual, Socratic feedback

42. beSocratic.chemistry.msu.edu

43. Summary  The time is right to implement evidence based practices in STEM education  Appropriate assessments are crucial  We are all in this together – collaboration is key