1. How People Learn Engineering: Two Examples and A Call to Action Cynthia J. Atman Director, Center for Engineering Learning and Teaching Associate Professor, Industrial Engineering University of Washington Richard Felder Hoechst Celanese Professor Emeritus of Chemical Engineering North Carolina State University Jennifer E. Turns Assistant Professor, Technical Communication Department University of Washington

2. Acknowledgments Grants: National Science Foundation (RED-9358516, NSF-9714459, EEC-9872498), GE Fund, Ford Motor Company Fund, Lockheed Martin, SUCCEED Coalition (EEC-9727411), Westinghouse Foundation, Xerox Corporation, University of Washington. Data analysis: Robin Adams (University of Washington), Mary Sacre, Heather Nachtmann and Justin Chimka (University of Pittsburgh), and teams of undergraduate and graduate students from the University of Washington, the University of Pittsburgh, and North Carolina State University.

3. How do engineering students gather information? How do engineering students solve engineering design problems? Do some teaching methods work better than others? Do they work better for all students or only certain types of learners? Do engineering graduates define problems broadly? Do engineering graduates integrate the concepts they are learning – linking them to each other? Thinking About Engineering Student Learning

4. Why are these important questions? We are surrounded by engineered artifacts that are getting more complex to design and maintain Engineering graduates need the skills to help them succeed in this environment immediately upon graduation and throughout their careers Engineering educators need to design educational experiences that develop these skills We need better information about how engineering students learn!

5. Can the Engineering Education Community Answer These Questions? Rene Magritte, c. 1956

6. Slice One: Design Processes Slice Two: Teaching and Learning A Call to Action: Need More Research Two Slices

7. How do engineering students gather information? How do engineering students solve engineering design problems? Slice One – Design Processes

8. You live in a mid-size city. A local resident has recently donated a corner lot for a playground. Since you are an engineer who lives in the neighborhood, you have been asked by the city to design a playground. You estimate that most of the children who will use the playground will range from 1 to 10 years of age...(other specifications)...Your design should use materials that are available at any hardware or lumber store. The playground must be ready for use in 2 months. Please take a moment and start a list of the information you would need to solve this problem. Playground Problem Statement

9. 26 entering students, 24 graduating students Graduating students from CE, IE and ME Solved “Playground Problem” talking out loud Asked experiment administrator for information as solved problem Took from 2 to 3 hours Experimental Design

10. Categories of Information Material costs Material specifications Neighborhood demographics Neighborhood opinions Safety Supervision concerns Technical references Utilities Availability of materials Body dimensions Budget Handicapped accessibility Information about the area Labor availability and costs Legal liability Maintenance concerns

11. Entering University Students 0255075100 Utilities Neighborhood opinions Legal liability Technical references Supervision concerns Availability of materials Other Maintenance concerns Handicapped accessibility Information about the area Neighborhood demographics Budget Labor availability and costs Body dimensions Safety Material costs Material specifications Categories Percent of subjects requesting information in each category Graduating University Students 0255075100 Utilities Neighborhood opinions Legal liability Technical references Supervision concerns Availability of materials Other Maintenance concerns Handicapped accessibility Information about the area Neighborhood demographics Budget Labor availability and costs Body dimensions Safety Material costs Material specifications Categories Percent of subjects requesting information in each category Results: Information Gathering Bursic, Karen M. and Cynthia J. Atman, “Information Gathering: A Critical Step for Quality in the Design Process,” Quality Management Journal, vol. 4, no. 4, pp. 60-75, 1997.

12. Identification of a Need Problem Definition Information Gathering Generation of Ideas Modeling Feasibility of analysis Evaluation Decision Communication Implementation Problem Scoping * Developing Alternative Solutions Project Realization* * Areas where recent reports say students need to improve. Design Process Activities (Derived from analysis of 7 engineering texts)

13. In our studies on design Subjects solve design problem while talking aloud Transcribe protocol Segment transcript into codable “chunks” of subject statements (reliability check) Code transcript (reliability check) Analyze to answer specific research questions Verbal Protocol Analysis

14. Results: Entering vs. Graduating Students Graduating students had higher quality designs (whew!!) gathered more information, covering more categories made more transitions among design steps spent more time iterating and iterate more effectively (Adams, 2001) progressed farther in the design process

15. Design Process Timelines 00:00:00:0000:16:00:0000:32:00:0000:48:00:0001:04:00:0001:20:00:0001:36:00:0001:52:00:0002:08:00:0002:24:00:00 PD GATH GEN MOD FEAS EVAL DEC COMM Successful Graduating Student (High Quality Score = 0.63) 00:00:00:0000:16:00:0000:32:00:0000:48:00:0001:04:00:0001:20:00:0001:36:00:0001:52:00:00 PD GATH GEN MOD FEAS EVAL DEC COMM Canonical Entering Student (Low Quality Score = 0.37)

16. Design Process Timelines Entering University Students Graduating University Students Atman, Cynthia J., Justin R. Chimka, Karen M. Bursic, and H. L. Nachtmann, “A Comparison of Freshman and Senior Engineering Design Processes,” Design Studies, vol. 20, no. 2, pp. 131-152, March 1999.

17. Do some teaching methods work better than others? Do they work better for all students or only certain types of learners? Slice Two – Teaching and Learning

18. A Longitudinal Study of Engineering Student Performance and Retention Teach five CHE courses in successive semesters using –inductive, active, cooperative learning –problems that address a variety of disciplines and thinking skills –lab & plant visits, exposure to practicing engineers –carefully designed tests, criterion- referenced grading

19. Collect –demographic data, precollege credentials, MBTI & LASSI profiles, first-year grades –academic performance and retention data from first CHE course through graduation –responses to survey questions about attitudes to courses, confidence levels, career goals Compare academic and affective outcomes for experimental group and traditionally- taught comparison group

20. Groups Studied Experimental Group –N = 123 –29% female, 71% male –6% Afr.-Amer., 5% Asian-Amer., 84% white, 5% other –Taught with method outlined in 5 courses Comparison Group –N = 190 –36% female, 64% male –7% Afr.-Amer., 10% Asian-Amer., 81% white, 2% other –Taught traditionally in all courses No between-group demographic differences statistically significant at.05 level.

21. Grade Distributions in Introductory Course p <.001 p =.02

22. Percent Giving Highest Rating to Quality of Preparation by Prerequisite Courses p=.16 p<.001 * No sequence courses among prerequisites ** Capstone design course

23. Seniors’ Ratings of the Classroom Environment p<.001

24. Seniors’ Ratings of CHE Instructional Quality p<.001

25. 5-Year Graduation and Attrition Rates* * “Grad” = graduated in chemical engineering “Left” = switched curricula or dropped out of school p<.01

26. Other Results Gender differences Rural/urban differences MBTI type effects Self-assessments of problem-solving abilities (basic & creative), self- confidence levels, post-graduation plans & career goals

27. We need more slices!!! A Call to Action

28. What do we know from research in other fields? Expert/novice differences Situated cognition Conceptual change Problem-based learning …….. A Platform Exists to Build From: Research Results

29. How do we leverage these results? Provide insights Provide vocabulary Help us to select areas likely to succeed …….. -> BUT, cannot replace research in engineering learning A Platform Exists to Build From : Leveraging Results

30. What is the nature of engineering expertise? How do engineers integrate knowledge and apply it to solve engineering problems? How do we adapt teaching innovations known to be successful in other domains to engineering? How do we motivate students? ……….. Moving into Engineering Education: Research Questions

31. Moving into Engineering Education: Multiple Approaches Example Study Designs Responses to teaching methods Responses by sub-populations (coop vs. non-coop,...) Detailed descriptions of behaviors, understandings, skills, attitudes by –Case studies –Observation or interview –Verbal protocol analysis Meta-analyses Statistical analyses of factors contributing to performance Issues Qualitative and quantitative Presence of control/ comparison groups Balancing size of populations with depth of analysis One shot vs. longitudinal studies Triangulation