1. Research on Undergraduate Learning in STEM Disciplines Karl A. Smith Civil Engineering University of Minnesota ksmith@umn.edu www.ce.umn.edu/~smith National Research Council National Science Resources Center Math/Science Partnerships Workshop December 5-7, 2004

2. Backdrop National Research Council Reports: 1.How People Learn: Brain, Mind, Experience, and School (1999). 2.How People Learn: Bridging Research and Practice (2000). 3.Knowing What Students Know: The Science and Design of Educational Assessment (2001). 4.The Knowledge Economy and Postsecondary Education (2002). Chapter 6 – Creating High-Quality Learning Environments: Guidelines from Research on How People Learn

5. Session Highlights Provide overview of some findings from reports related to teaching & learning. Do a activities with you to illustrate some of the points covered in the reports. Discuss implications for designing learning environments that are learner centered, knowledge centered, assessment centered, and community centered.

6. Designing Learning Environments Based on HPL

7. Learner-Centered Learning Environments

10. Knowledge-Centered Learning Environments

11. Assessment-Centered Learning Environments

13. Community-Centered Learning Environments

14. Summary Points There is an emerging science of learning It has major implications for all aspects of schooling -- curriculum, instruction, assessment, plus preservice and inservice teacher education It provides a basis for knowing when, how and why to use various instructional strategies It can guide the intelligent design and use of new curricular materials as well as information technologies

15. Lila M. Smith

16. Pedago-pathologies B Lee Shulman Amnesia Fantasia Inertia Shulman, Lee S. 1999. Taking learning seriously. Change, 31 (4), 11-17.

17. What do we do about these pathologies? – Lee Shulman Activity Reflection Collaboration Passion Combined with generative content and the creation of powerful learning communities Shulman, Lee S. 1999. Taking learning seriously. Change, 31 (4), 11-17.

18. Lila M. Smith

19. Tracking Change - Seymour "The greatest single challenge to SMET pedagogical reform remains the problem of whether and how large classes can be infused with more active and interactive learning methods." Seymour, Elaine. 2001. Tracking the processes of change in US undergraduate education in science, mathematics, engineering, and technology. Science Education, 86, 79-105.

20. Formulate-Share-Listen-Create (Think-Pair-Share) Individually read the quote “To teach is to engage students in learning...” Underline/Highlight words and/or phrase that stand out for you Turn to the person next to you, introduce yourself Share words and/or phrases that stood out and discuss

21. To teach is to engage students in learning; thus teaching consists of getting students involved in the active construction of knowledge...The aim of teaching is not only to transmit information, but also to transform students from passive recipients of other people's knowledge into active constructors of their own and others' knowledge...Teaching is fundamentally about creating the pedagogical, social, and ethical conditions under which students agree to take charge of their own learning, individually and collectively Education for judgment: The artistry of discussion leadership. Edited by C. Roland Christensen, David A. Garvin, and Ann Sweet. Cambridge, MA: Harvard Business School, 1991.

22. Strategies for Energizing Large Classes: From Small Groups to Learning Communities: Jean MacGregor, James Cooper, Karl Smith, Pamela Robinson New Directions for Teaching and Learning, No. 81, 2000. Jossey- Bass

23. Book Ends on a Class Session

24. Informal CL (Book Ends on a Class Session) with Concept Tests Physics Peer Instruction Eric Mazur - Harvard B http://galileo.harvard.edu Peer Instruction – www.prenhall.com Richard Hake – http://www.physics.indiana.edu/~hake/ Chemistry Chemistry ConcepTests - UW Madison B www.chem.wisc.edu/~concept Video: Making Lectures Interactive with ConcepTests ModularChem Consortium B http://mc2.cchem.berkeley.edu/ STEMTEC Video: How Change Happens: Breaking the ATeach as You Were Taught@ Cycle B Films for the Humanities & Sciences B www.films.com Thinking Together video: Derek Bok Center B www.fas.harvard.edu/~bok_cen/

25. Richard Hake (Interactive engagement vs traditional methods) http://www.physics.indiana.edu/~hake/ Traditional (lecture) Interactive (active/cooperative) = Concept Inventory Gain/Total

27. The “Hake” Plot of FCI Pretest (Percent) 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 20.0030.0040.0050.0060.0070.0080.00 ALS SDI WP PI(HU) ASU(nc) ASU(c) HU WP* UMn Traditional X UMn Cooperative Groups X UMn-CL+PS

28. Physics (Mechanics) Concepts: The Force Concept Inventory (FCI) A 30 item multiple choice test to probe student's understanding of basic concepts in mechanics. The choice of topics is based on careful thought about what the fundamental issues and concepts are in Newtonian dynamics. Uses common speech rather than cueing specific physics principles. The distractors (wrong answers) are based on students' common inferences.

29. FCI Question 17 An elevator is being lifted up an elevator shaft at a constant speed by a steel cable, as shown in the figure. All frictional effects are negligible. In this situation, forces on the elevator are such that: (A)the upward force by the cable is greater than the downward force of gravity. (B)the upward force by the cable is equal to the downward force of gravity. (C)the upward force by the cable is smaller than the down ward force of gravity. (D)the upward force by the cable is greater than the sum of the downward force of gravity and a downward force due to the air. (E)None of the above. (The elevator goes up because the cable is shortened, not because an upward force is exerted on the elevator by the cable). Pre 64 18 2 11 5 Post 36 60 0 2 1

30. Problem Based Cooperative Learning Format TASK: Solve the problem(s) or Complete the project. INDIVIDUAL: Estimate answer. Note strategy. COOPERATIVE: One set of answers from the group, strive for agreement, make sure everyone is able to explain the strategies used to solve each problem. EXPECTED CRITERIA FOR SUCCESS: Everyone must be able to explain the strategies used to solve each problem. EVALUATION: Best answer within available resources or constraints. INDIVIDUAL ACCOUNTABILITY: One member from your group may be randomly chosen to explain (a) the answer and (b) how to solve each problem. EXPECTED BEHAVIORS: Active participating, checking, encouraging, and elaborating by all members. INTERGROUP COOPERATION: Whenever it is helpful, check procedures, answers, and strategies with another group.

31. Technical Estimation Exercise TASK: INDIVIDUAL: Quick Estimate (10 seconds). Note strategy. COOPERATIVE: Improved Estimate (~5 minutes). One set of answers from the group, strive for agreement, make sure everyone is able to explain the strategies used to arrive at the improved estimate. EXPECTED CRITERIA FOR SUCCESS: Everyone must be able to explain the strategies used to arrive at your improved estimate. EVALUATION: Best answer within available resources or constraints. INDIVIDUAL ACCOUNTABILITY: One member from your group may be randomly chosen to explain (a) your estimate and (b) how you arrived at it. EXPECTED BEHAVIORS: Active participating, checking, encouraging, and elaborating by all members. INTERGROUP COOPERATION: Whenever it is helpful, check procedures, answers, and strategies with another group.

32. Model 1 (lower bound) let L be the length of the room, let W be its width, let H be its height, and let D be the diameter of a ping pong ball. Then the volume of the room is V room = L * W * H, and the volume of a ball (treating it as a cube) is V ball = D 3, so number of balls = (V room ) / (V ball ) = (L * W * H) / (D 3 ).

33. Model 2 (upper bound) let L be the length of the room, let W be its width, let H be its height, and let D be the diameter of a ping pong ball. Then the volume of the room is V room = L * W * H, and the volume of a ball (treating it as a sphere) is V ball = 4/3 Br 3, so number of balls = (V room ) / (V ball ) = (L * W * H) / (4/3 Br 3 ).

34. Model 1 (V room / D 3 ball ) B Lower Bound Model 2 (V room / (4/3 Br 3 ball )) B Upper Bound Upper Bound/Lower Bound = 6/B. 2 How does this ratio compare with 1.The estimation of the diameter of the ball? 2.The estimation of the dimensions of the room?

35. Real World Model World Model V r /V b Calc

37. Problem-Based Learning Problem posed Identify what we need to know Learn it Apply it START Subject-Based Learning Told what we need to know Learn it Given problem to illustrate how to use it START Normative Professional Curriculum: 1.Teach the relevant basic science, 2.Teach the relevant applied science, and 3.Allow for a practicum to connect the science to actual practice.

38. Problem-Based Learning (PBL) Problem-based learning is the learning that results from the process of working toward the understanding or resolution of a problem. The problem is encountered first in the learning process B Barrows and Tamlyn, 1980 Core Features of PBL Learning is student-centered Learning occurs in small student groups Teachers are facilitators or guides Problems are the organizing focus and stimulus for learning Problems are the vehicle for the development of clinical problem- solving skills New information is acquired through self-directed learning

39. Group Processing B Plus/Delta Format B Plus Things That Group Did Well Delta Things Group Could Improve

40. Cooperative Learning is instruction that involves people working in teams to accomplish a common goal, under conditions that involve both positive interdependence (all members must cooperate to complete the task) and individual and group accountability (each member is accountable for the complete final outcome). Key Concepts Positive Interdependence Individual and Group Accountability Face-to-Face Promotive Interaction Teamwork Skills Group Processing

41. Cooperative Learning Research Support Johnson, D.W., Johnson, R.T., & Smith, K.A. 1998. Cooperative learning returns to college: What evidence is there that it works? Change, 30 (4), 26-35. Over 300 Experimental Studies First study conducted in 1924 High Generalizability Multiple Outcomes Outcomes 1. Achievement and retention 2. Critical thinking and higher-level reasoning 3. Differentiated views of others 4. Accurate understanding of others' perspectives 5. Liking for classmates and teacher 6.Liking for subject areas 7. Teamwork skills

42. Small-Group Learning: Meta-analysis Springer, L., Stanne, M. E., & Donovan, S. 1999. Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta- analysis. Review of Educational Research, 69(1), 21-52. Small-group (predominantly cooperative) learning in postsecondary science, mathematics, engineering, and technology (SMET). 383 reports from 1980 or later, 39 of which met the rigorous inclusion criteria for meta-analysis. The main effect of small-group learning on achievement, persistence, and attitudes among undergraduates in SMET was significant and positive. Mean effect sizes for achievement, persistence, and attitudes were 0.51, 0.46, and 0.55, respectively.

43. Creating High-Quality Learning Environments: Guidelines from Research on How People Learn Understanding by Design Wiggins & McTighe Backward Design Stage 1.Identify Desired Results Stage 2.Determine Acceptable Evidence Stage 3.Plan Learning Experiences and Instruction Wiggins, G. & McTighe, J. 1998. Understanding by design. ASCD.

44. Backward Design Stage 1. Identify Desired Results Filter 1. To what extent does the idea, topic, or process represent a big idea or having enduring value beyond the classroom? Filter 2. To what extent does the idea, topic, or process reside at the heart of the discipline? Filter 3. To what extent does the idea, topic, or process require uncoverage? Filter 4. To what extent does the idea, topic, or process offer potential for engaging students?

45. Backward Design Stage 2. Determine Acceptable Evidence Types of Assessment Quiz and Test Items: Simple, content-focused test items Academic Prompts: Open-ended questions or problems that require the student to think critically Performance Tasks or Projects: Complex challenges that mirror the issues or problems faced by graduates, they are authentic

46. Backward Design Stage 3. Plan Learning Experiences & Instruction What enabling knowledge (facts, concepts, and principles) and skills (procedures) will students need to perform effectively and achieve desired results? What activities will equip students with the needed knowledge and skills? What will need to be taught and coached, and how should it be taught, in light of performance goals? What materials and resources are best suited to accomplish these goals? Is the overall design coherent and effective?

47. It could well be that faculty members of the twenty-first century college or university will find it necessary to set aside their roles as teachers and instead become designers of learning experiences, processes, and environments James Duderstadt, 1999

48. We never educate directly, but indirectly by means of the environment. Whether we permit chance environments to do the work, or whether we design environments for the purpose makes a great difference. John Dewey, 1906

49. CAEE Vision for Engineering Education Center for the Advancement of Engineering Education Cindy Atman, Director

50. CAEE Team University of Washington Colorado School of Mines Howard University Stanford University University of Minnesota CAEE Affiliate Organizations City College of New York (CCNY), Edmonds Community College, Highline Community College (HCC), National Action Council for Minorities in Engineering (NACME), North Carolina A&T (NCA&T), San Jose State University (SJSU), University of Texas, El Paso (UTEP), Women in Engineering Programs & Advocates Network (WEPAN) and Xavier University

51. CAEE - Elements for Success Scholarship on Learning Engineering Learn about the engineering student experience Scholarship on Engineering Teaching Help faculty improve student learning Scholarship on Engineering Education Institutes Cultivate future leaders in engineering education

52. Theory Research Practice CAEE Approach Research that makes a difference... in theory and practice

53. Center for the Integration of Research, Teaching, and Learning (CIRTL) NSF Center for Learning and Teaching University of Wisconsin - Madison Michigan State University Pennsylvania State University

54. …develop a national STEM faculty... Research Universities 100 RUs => 80% Ph.D’s FACULTY Community College Liberal Arts HBCU Masters University Comprehensive Univ. Research University UNDERGRADS Community College Liberal Arts HBCU Masters University Comprehensive Univ. Research University

55. Teaching-as-Research Engagement in teaching as engagement in STEM research Engagement in teaching as engagement in STEM research Hypothesize, experiment, observe, analyze, improve Hypothesize, experiment, observe, analyze, improve Aligns with skills and inclinations of graduates- Aligns with skills and inclinations of graduates- through-faculty, and fosters engagement in through-faculty, and fosters engagement in teaching reform teaching reform Leads to self-sustained improvement of STEM education Leads to self-sustained improvement of STEM education “The nation must develop STEM faculties who themselves continuously inquire into their students’ learning.”

56. NATIONAL ACADEMY OF ENGINEERING OF THE NATIONAL ACADEMIES Center for the Advancement of Scholarship on Engineering Education 56 A Work-in-Progress: NAE Center for the Advancement of Scholarship on Engineering Education Norman L. Fortenberry, Sc.D. Director, CASEE http://www.nae.edu/CASEE nfortenb@nae.edu (202) 334-1926 November 8, 2003

57. NATIONAL ACADEMY OF ENGINEERING OF THE NATIONAL ACADEMIES Center for the Advancement of Scholarship on Engineering Education 57 CASEE Mission Enable engineering education to meet, in a significantly better way, the needs of employers, educators, students, and society at large. Working collaboratively with key stakeholders, CASEE l Encourages rigorous research on all elements of the engineering education system, and l Seeks broad dissemination, adoption, and use of research findings. CASEE Objectives

58. NATIONAL ACADEMY OF ENGINEERING OF THE NATIONAL ACADEMIES Center for the Advancement of Scholarship on Engineering Education 58 Research Thrust Areas 1. Define the bodies-of-knowledge required for engineering practice and use of engineering study for other careers. 2. Develop strategies that value diversity in the formulation and solution of engineering problems. 3. Develop cost-effective and time-efficient strategies and technologies for Improving student learning, and Enhancing the instructional effectiveness of current and future faculty. 4. Develop assessments of student learning and instructional effectiveness.

59. Conducting Rigorous Research in Engineering Education: Creating a Community of Practice NSF-CCLI-ND American Society for Engineering Education Karl Smith & Ruth Streveler University of Minnesota & Colorado School of Mines

60. Rigorous Research Workshop  Initial Event for year-long project  Presenters and evaluators representing –American Society for Engineering Education (ASEE) –American Educational Research Association (AERA) –Professional and Organizational Development Network in Higher Education (POD)  Faculty funded by two NSF projects: –Conducting Rigorous Research in Engineering Education (NSF DUE- 0341127) –Strengthening HBCU Engineering Education Research Capacity (NSF HRDF-041194) Council of HBCU Engineering Deans Center for the Advancement of Scholarship in Engineering Education (CASEE) National Academy of Engineering (NAE)