1. Teaching Students How to Evaluate Calculation Results James Hanson, Ph.D, PE Rose-Hulman Institute of Technology ASEE Illinois-Indiana Section Conference March 27, 2009

2. Overview Motivation How evaluation is done How to teach it Reinforcing it in HW How well it works

3. Motivation Not in textbooks Reliance on computers Experience is retiring!

4. How Evaluation is Done Interview 35 practitioners Ten firms Experience: 1-55 years Study Performed

5. How Evaluation is Done Fundamental Principles Approximations Features of the Solution Categories of Evaluation Tools for Analysis Results

6. How to Teach It Structural Engineering Electrical Engineering Other Examples

7. Fundamental Principles (for Structural Engineering) Statics –Equilibrium always Mechanics of Materials –Strain distribution –Stiffness attracts load

8. Fundamental Principles Example

9. Approximations (for Structural Engineering) Simplify loading Simplify geometry Assume material behavior

10. Approximations Example L/4 P/3 M max = PL/6 Actual L  = P/L M max = PL/8 Approximate Difference = – 25%

11. Features of the Solution (for Structural Engineering) Boundary conditions Continuity Fundamental principles Anticipate graphical features Bound possible solution To… Use…

12. Features of Solution Example C T = – M T C = + M + – M Construct the moment diagram: 1.Sketch displaced shape 2.Deduce sign of moment from curvature 3.Construct diagram

13. Electrical Engineering Examples Fundamental Principles: Circuit analysis governed by Ohm’s Law and Kirchoff’s Current Law Approximations: DC circuit with capacity approximated as open circuit DC circuit with inductor approximated as a short Features of the Answer: Ripples in frequency response of filter relates to order of the filter Thanks to Dr. Yoder, ECE Dept, RHIT

14. How to Teach It Fundamental Principles Approximations Features of the Solution Other Examples

15. How to Teach It Def: Def: Sequence of steps followed by a person to monitor and improve that person’s own cognitive performance in an area How: How: Set up student so intuition leads to wrong conclusion, then ask to reflect on why went wrong Metacognition

16. Metacognition (a) (b) Which way will right end deflect (up or down)?

17. a)Guess part of the solution (instructor choice) b)Generate approximate solution (text prob?) c)Use computer for solution d)Identify expected features in computer solution e)Verify fundamental principles in computer solution a)Guess part of the solution (instructor choice) approximate b)Generate approximate solution (text prob?) c)Use computer for solution features d)Identify expected features in computer solution fundamental principles e)Verify fundamental principles in computer solution f)Make comprehensive argument that computer result is reasonable g)Compare guess with solution and reflect on why similar/different Reinforcing It in HW

18. Reinforcing It in HW Example Problem Statement: An indeterminate beam experiences uniformly distributed load. Objective: Find vertical reactions.

19. Reinforcing It in HW Example a) Guess the deflected shape Not graded based on correctness, graded based on whether done

20. Reinforcing It in HW Example b) Generate approximate solution Assumption: load carried by nearest support

21. Reinforcing It in HW Example c) Use computer for solution Can use as “black box” if teaching how to evaluate results

22. Reinforcing It in HW Example d) Identify expected features in computer solution Displaced shape features Internal force diagram features Reaction directions

23. Reinforcing It in HW Example e) Verify fundamental principles in computer solution Check equilibrium:  F x = 0  F y = 0  M = 0

24. Reinforcing It in HW Example f) Make comprehensive argument that computer results are reasonable Features of Solution Fundamental Principles Approximate Result √ √ √

25. Reinforcing It in HW Example g) Compare guess with solution and reflect on why similar/different Graded based on candor and depth of reflection, not how similar or different

26. How Well It Works Attitude Undergrads should be able to determine reasonableness of results 2004 2005 2006 2007 Average 4 – Strongly Agree1 – Strongly Disagree YearStatement 30 46 42 29 n (ctrl) 2008 3.1 3.3 3.6 3.5 3.3 39 3.0 3.1 Metacognition improved performance in this course 2006 2007 33 23 200832

27. How Well It Works Behavior AffirmativeYearStatementn I use metacognition more now than before course 2006 2007 42 30 2008 74% 78% 40

28. How Well It Works Behavior 3.5 5 – Almost Always1 – Never AverageStatementn (Respond 1 yr after course) I use fundamental principles, approx, and features of soln to evaluate in other areas of civil 2006 39 3.6 I now use metacognition … than before the course 2006 39 5 – Much More1 – Much Less

29. Correctly Identify Most Reasonable How Well It Works Cognitive Fall 2004 (Ctrl) Fall 2005 Fall 2006 Fall 2007 Correctly Explain Why Reasonable Term 34 48 44 30 n Fall 200842 Structural Mechanics I Course (Required) Practitioners 49% 70% 69% 73% 72% 50% 67% 66% 69% 92%8

30. Correctly Identify Most Reasonable How Well It Works Cognitive Winter 2004 (Ctrl) Winter 2005 Winter 2006 Winter 2007 Correctly Explain Why Reasonable Term 9 9 17 11 n Winter 200816 Structural Mechanics II Course (Elective) Practitioners 46% 44% 65% 71% 68% 58% 35% 43% 49% 48% 59% 65%8

31. Summary Evaluation of results can be taught Fundamental principles, approximations, features of the solution Metacognition might help

32. Acknowledgements Participating Firms Sponsor Grant: DUE-0341212 Questions or comments: james.hanson@rose-hulman.edu Center for Structural Engineering Education http://www.rose-hulman.edu/csee