CREATE Mini Conference Melanie M Cooper Michigan State University May 2013

Todays students  Will have on average > 8 jobs  May have jobs that have not been invented yet  (~ 65% of them according to NYT)  Need skills that are not necessarily emphasized in college

The time is right They could drive significant improvements in STEM education. Proposed changes to both K-12 and college STEM education are upon us

BUT, K-12 and higher education have a chicken and egg problem

Students arrive at college underprepared College faculty blame K-12 system But teachers learn content in our STEM courses Methods classes don’t help if not integrated with appropriate content Teachers teach what and how they were taught STEM content There is a huge need for STEM disciplinary departments to engage in education research and practice

PCAST agrees!

PCAST Recommendations 1.Catalyze widespread adoption of empirically validated teaching practices. 2.Advocate and provide support for replacing standard laboratory courses with discovery based research courses. 3.Launch a national experiment in postsecondary mathematics education to address the math preparation gap. 4.Encourage partnerships among stakeholders to diversify pathways to STEM careers. 5.Create a Presidential Council on STEM Education to-excel-final_feb.pdf

PCAST Recommendations 1.Catalyze widespread adoption of empirically validated teaching practices. 2.Advocate and provide support for replacing standard laboratory courses with discovery based research courses. 3.Launch a national experiment in postsecondary mathematics education to address the math preparation gap. 4.Encourage partnerships among stakeholders to diversify pathways to STEM careers. 5.Create a Presidential Council on STEM Education to-excel-final_feb.pdf

PCAST Recommendation 1 1.1Establish discipline-focused programs funded by Federal research agencies, academic institutions, disciplinary societies, and foundations to train current and future faculty in evidence- based teaching practices. 1.2Create a “STEM Institutional Transformation Awards” competitive grants program at NSF 1.3Request that the National Academies develop metrics to evaluate STEM education

1.1 Evidence-Based Teaching Practices?

Discipline Based Education Research “DBER combines knowledge of teaching and learning with deep knowledge of discipline based STEM content. It describes the specific difficulties learners face and the specialized intellectual and instructional resources that can facilitate student understanding”

Can Impact How we teachHow students learn What we teachThe curriculum Who we teach Issues of access and diversity How we know students have learned Assessment DBER

Focus on College level Discipline Specific Used by College Faculty Science Education Focus on K-12 Used by trained teachers Cognitive/Educational Psychology Focus on learning Not discipline specific SoTL Focus on College level Often inter- disciplinary But really these categories are fuzzy – lots of overlap How is DBER different than other areas of research in teaching and learning?

The NRC report will provide focus and future directions for the growing field of DBER

To meet the challenges involved in improving STEM education, collaboration is the KEY

1.2Create a “STEM Institutional Transformation Awards” competitive grants program at NSF

1.3Request that the National Academies develop metrics to evaluate STEM education This is where the rubber hits the road!

“to educate and improve student performance, not merely to audit it” Assessment: Wiggins, G. (1998). Educative assessment: Designing assessments to inform and improve student performance. San Francisco, CA: Jossey-Bass “if you don’t assess what’s important, what’s assessed becomes important”

Our ultimate goal: to allow students to learn the knowledge and skills that can be applied to new situations Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century, NRC 2012

How to assess/evaluate what we say we value is a major issue

A cautionary tale Year%DFWACS %ile# students# DFW

Reforms  Weekly meetings to negotiate “big ideas” and learning outcomes and assessments (backward design)  Reduce class size (to about 100 from 180)  Remove content (~30%)  Add “active” learning (group work, clickers etc)  Each faculty member uses their own notes/class management style  there were typically no difference in grade distribution regardless of amount of “active learning”

Success! Year%DFWACS %ile# students# DFW We can go home

Not so fast… What happens to the potential energy when you bring two hydrogen atoms together? “The potential energy goes up … when you break a bond it releases energy”

Energy changes and bonding % Students by type with bond energy misconceptions Chemistry level (#)% General Chemistry (77)50 Inorganic (13)54 Organic (172)65 Analytical (35)51 Physical (16)56 Graduate Students (21)68 Post docs (80)68

In our studies we found that good students (hard working, conscientious, bright A students) had significant problems understanding core disciplinary ideas, and were unable to apply their knowledge to new situations

NRC Report: Assessing 21 st century skills

21 st Century Skills  Cognitive skills: nonroutine problem solving, critical thinking, systems thinking  Interpersonal skills: complex communication, social skills, teamwork, cultural sensitivity, dealing with diversity  Intrapersonal skills: self-management, time management, self development, self-regulation, adaptability, executive functioning

NRC report: Assessing 21 st Century Skills (2011) Economy-wide measures of routine and nonroutine task input: (1969 = 0). SOURCE: Levi and Murnane (2004).

The Next Generation Science Standards (NGSS) provide us with a potential model to drive development of new assessments

Scientific and Engineering Practices 1. Asking questions and defining problems 2. Developing and using models 3. Planning and carrying out investigations and design solutions 4. Analyzing and interpreting data 5. Using mathematics and computational thinking 6. Developing explanations and designing solutions 7. Engaging in argument from evidence 8. Obtaining, evaluating, and communicating information The multiple ways of knowing and doing that scientists and engineers use to study the natural world and design world.

Performance Expectation Cross Cutting Science Practices Core Content NGSS Performance Expectations drive assesment

HS-PS1-4 “Develop a model to illustrate that the release or absorption of energy from a chemical reaction system depends upon the changes in total bond energy.” Performance Expectations

HS-PS-1c. Plan and conduct an investigation to gather evidence to compare the structure of substances at the bulk scale to infer the electrical forces within and between atoms.

How many molecules are in 1.00 kg of hydrazine, N 2 H 4 ? A.9.40  B.1.88  C.1.88  D.1.13  E.1.13  Not this

But This

Use technology to design, administer, grade, model and evaluate student performances

Problem Solving Strategies -captured during student performance Stevens, R., Soller, A., Cooper, M., Sprang, M., (2004) Computer Science Editorial III, Springer- Verlag, Heidelberg Germany

Assessment of Problem Solving IMMEX strategy (ANN HMM) (concurrent) IMMEX ability (IRT) (concurrent) Metacognitive Activity(MCAi) (pre/post Cooper, M. M.; Sandi-Urena, S., Chem. Educ. Res. Pract., 2008, 9, Across method and time.

beSocratic  Allows, recognizes and responds to free form input – graphs, chemical structures, diagrams, gestures, and some text  Provides tiered, contextual, Socratic feedback

beSocratic.chemistry.msu.edu

Summary  The time is right to implement evidence based practices in STEM education  Appropriate assessments are crucial  We are all in this together – collaboration is key