1. A Master Thesis Presented by Frank Diete, M.S. IEOR Evaluation of a Simulator Based, Novice Driver Risk Awareness Training Program 27th September, 2007 Committee: Prof. Donald Fisher, Prof. John Collura, Prof. Sundar Krishnamurty

2. 2 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Outline  Review of Motivation  Experimental Design Training Experimental Group Training Control Group Simulator Evaluation  Scoring and Results Training Results HPL Simulator Results  Conclusion

3. 3 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Young, novice drivers are overrepresented in car crashes Background and Motivation (Insurance Institute for Highway Safety (2006). Fatality Facts 2005: Older people. Retrieved July 6, 2007, from http://www.iihs.org/research/fatality_facts/olderpeople.html#sec0)

4. 4 Master Thesis Defense, presented by Frank Diete, 09.27.2007 The reasons are failures in search Field Studies Police Accidents Reports (Mc Knight & Mc Knight, 2003) Eye Tracking Data - Scanning Patterns (Mourant and Rockwell (1972), Crundall & Underwood (1998)) Simulator Study (Pradhan et al. (2005)*) Novice drivers do not show an appropriate tactical scanning behavior Failures in Selective Attention Pradhan et al. (2005)* *Pradhan, A.K., Hammel, K.R., DeRamus, R., Pollatsek, A., Noyce, D.A., & Fisher, D.L. (2005). The Use of Eye Movements to Evaluate the Effects of Driver Age on Risk Perception in an Advanced Driving Simulator. Human Factors, 47, 840-852.

5. 5 Master Thesis Defense, presented by Frank Diete, 09.27.2007 RAPT-1 was developed and evaluated on the HPL driving simulator by Pollatsek et al. (2006)*  RAPT does … … train novice drivers where to look  RAPT does not … … train manual skills … train time sharing skills Risk Awareness and Perception Training (RAPT) After training 90% of critical areas in plan views could be identified by participants. *Pollatsek, A., Narayanaan, V., Pradhan, A., & Fisher, D.L. (2006b). The use of eye movements to evaluate the effect of PC-based risk awareness training on an advanced driving simulator. Human Factors, 48, 447–464.

6. 6 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Simulator Evaluation Results Near Transfer Scenarios (51.9% vs. 27.3%) Far Transfer Scenarios (63.5% vs. 43.5%) Pollatsek et al., 2006* Total Effect of Training (57.7% vs. 35.4%) Although 90% of risky areas could be identified in the training, only 52% of these areas were fixated in the evaluation! What explains this big difference in performance? 1)Difficulty to generalize from plan views to a perspective view as seen on the driving simulator 2)Necessity of practicing hazard anticipation at the same time as driving  Multitasking does not occur automatically! 22.3% 20.0% 24.6% *Pollatsek, A., Narayanaan, V., Pradhan, A., & Fisher, D.L. (2006b). The use of eye movements to evaluate the effect of PC-based risk awareness training on an advanced driving simulator. Human Factors, 48, 447–464.

7. 7 Master Thesis Defense, presented by Frank Diete, 09.27.2007 A training program using both RAPT and the Drive Square simulator might result in better training effects. Link to proposed study + = RAPT*SIMRAPT Better Hazard Anticipation? *Pollatsek, A., Narayanaan, V., Pradhan, A., & Fisher, D.L. (2006b). The use of eye movements to evaluate the effect of PC-based risk awareness training on an advanced driving simulator. Human Factors, 48, 447–464.

8. 8 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Research Hypotheses  Hypothesis 1: A training program using a combination of plan views (RAPT) and a head mounted driving simulator (SIMRAPT) will result in trained participants recognizing risks significantly more often on the HPL driving simulator than untrained novice drivers. The effects of training will generalize from the combined RAPT and SIMRAPT training program to scenarios on the HPL driving simulator that measure both the near and far transfer of the ability to recognize risks.  Hypothesis 2: The difference in performance between trained and untrained novice drivers using the combined RAPT/SIMRAPT training program will be larger than in past studies which used only the RAPT training program where performance in this and previous studies was evaluated on the HPL driving simulator using the same scenarios. Experiment

9. 9 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Outline  Review of Motivation  Experimental Design Training Experimental Group Training Control Group Simulator Evaluation  Scoring and Results Training Results HPL Simulator Results  Conclusion

10. 10 Master Thesis Defense, presented by Frank Diete, 09.27.2007 The effects of the training were evaluated by comparing simulator performance of a trained experimental and an pseudo-trained control group Experimental Group RAPT-1 SIMRAPT (12 participants) HPL Simulator Evaluation Control Group Mass RMV Pseudo-SIMRAPT (12 participants) HPL Simulator Evaluation Compare Participants  16 or 17 years old  Less than 6 month driving experience Experiment Design

11. 11 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Participants Experimental Group 10 permit drivers 2 license drivers 7 male 5 female Average Age: 16.5 years STDV: 0.4 years Control Group 10 permit drivers 2 license drivers 7 male 5 female Average Age: 16.5 years STDV: 0.5 years

12. 12 Master Thesis Defense, presented by Frank Diete, 09.27.2007 The effects of the training were evaluated by comparing simulator performance of a trained experimental and an pseudo-trained control group Experimental Group RAPT-1 SIMRAPT (12 participants) HPL Simulator Evaluation Control Group Mass RMV Pseudo-SIMRAPT (12 participants) HPL Simulator Evaluation Compare Participants  16 or 17 years old  Less than 6 month driving experience Experiment Design

13. 13 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Training Program: Experimental Group RAPT Instructions Pre-Test / Training on 8 scenarios Post-Test SIMRAPT The slides for the RAPT PowerPoint were developed by Anuj Pradhan for prior studies.

14. 14 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Training Program: Experimental Group RAPT SIMRAPT Practice Drive Drive 1st scenario Present plan view And feedback End training yes Drive next scenario no yes no Repeat scenario no Driver feels comfortable Correct head-turn done? All scenarios driven? Written Instructions

15. 15 Master Thesis Defense, presented by Frank Diete, 09.27.2007 The effects of the training were evaluated by comparing simulator performance of a trained experimental and an pseudo-trained control group Experimental Group RAPT-1 SIMRAPT (12 participants) HPL Simulator Evaluation Control Group Mass RMV Pseudo-SIMRAPT (12 participants) HPL Simulator Evaluation Compare Participants  16 or 17 years old  Less than 6 month driving experience Experiment Design

16. 16 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Training Program: Control Group Mass RMV Training  Nothing related to hazard anticipation Drive Square Simulator

17. 17 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Training Program: Control Group Mass RMV Training Drive Square Simulator Practice Drive Drive all 8 scenarios one after another yesno Driver feels comfortable Drive all 8 scenarios again in another order End training No feedback given Written Instructions

18. 18 Master Thesis Defense, presented by Frank Diete, 09.27.2007 The effects of the training will be evaluated by comparing simulator performance of a trained experimental and an pseudo-trained control group Experimental Group RAPT-1 SIMRAPT (12 participants) HPL Simulator Evaluation Control Group Mass RMV Pseudo-SIMRAPT (12 participants) HPL Simulator Evaluation Compare Participants  16 or 17 years old  Less than 6 month driving experience Experiment Design

19. 19 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Both the experimental and the control group were evaluated on the HPL driving simulator  Evaluation immediatly after the training  16 Scenarios in four blocks  8 near and 8 far transfer scenarios*  Instructions  „obey traffic laws etc.“…  Participants are asked to follow a lead vehicle  Calibration of the eye-tracker  Practice Drive HPL simulator evaluation *The scenarios for the simulator evaluation had been developed by Anuj Pradhan at the Human Performance Lab for prior studies.

20. 20 Master Thesis Defense, presented by Frank Diete, 09.27.2007 The study in a movie

21. 21 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Outline  Review of Motivation  Experimental Design Training Experimental Group Training Control Group Simulator Evaluation  Scoring and Results Training Results HPL Simulator Results  Conclusion

22. 22 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Results RAPT Training

23. 23 Master Thesis Defense, presented by Frank Diete, 09.27.2007 SIMRAPT Results – Experimental Group Sequences of scenarios in SIMRAPT: 1st participant: 1 – 2 – 3 – 4 – 5 – 6 – 7 – 8 Scoring in SIMRAPT: 0, 0.5 or 1 point for each scenario and drive Number of trials counted for each scenario and drive 2nd participant: 2 – 3 – 4 – 5 – 6 – 7 – 8 – 1 3rd participant: 3 – 4 – 5 – 6 – 7 – 8 – 1 – 2 …..

24. 24 Master Thesis Defense, presented by Frank Diete, 09.27.2007 SIMRAPT Results – Experimental Group Average Score first trialAverage number of trials 0% 79% 2.58 1.33

25. 25 Master Thesis Defense, presented by Frank Diete, 09.27.2007 SIMRAPT Results – Experimental Group Average Score first trialAverage number of trials

26. 26 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Drive Square Results – Control Group Average Score 1st and 2nd run by scenario. Slight improvement from 8.9% to 14.1%, but not significant (t=1.93, p>.05)

27. 27 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Conclusion of training results  High and significant training effect for the experimental group from the 1st to the 8th drive on SIMRAPT  Generalization within training  Tendency of small training effect for control group from 1st to 2nd run on Drive Square

28. 28 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Outline  Review of Motivation  Experimental Design Training Experimental Group Training Control Group Simulator Evaluation  Scoring and Results Training Results HPL Simulator Results  Conclusion

29. 29 Master Thesis Defense, presented by Frank Diete, 09.27.2007 HPL Simulator Evaluation - Scoring Each scenario* was scored 1 or 0 depending on eye fixation behavior! *The scenarios for the simulator evaluation had been developed by Anuj Pradhan at the Human Performance Lab for prior studies.

30. 30 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Near Transfer – Opposing Truck Left Turn scenario Risk RecognizedRisk Not Recognized *The scenarios for the HPL driving simulator that are used for this study and the slides for the Power Point Training had been developed by Anuj Pradhan at the Human Performance Lab within the last years.

31. 31 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Far Transfer – Mullins Center scenario Risk RecognizedRisk Not Recognized *The scenarios for the HPL driving simulator that are used for this study and the slides for the Power Point Training had been developed by Anuj Pradhan at the Human Performance Lab within the last years.

32. 32 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Results for simulator evaluation Test of Hypothesis 1: Significant training effect of combined training program? T-Intersection Amity-Lincoln Left Fork Opposing Truck Left Turn Pedestrians on Left Truck Crosswalk Adjacent Truck Left Turn Blind Drive Intersection with One-Way Street Mullins Center Bus Left Turn at Triangle St Signal Ahead at hill Right on Red Curved Stop Ahead Truck Blocking Travel in Lane Intersection with new green All 16 scenarios included in evaluation!

33. 33 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Results for simulator evaluation ExperimentalControlDifferenceSignificance Near Transfer72.2%37.5%34.7%t(18)=3.1, p<.01 Far Transfer72.5%57.3%15.2%t(20)=2.1, p=.052 Total72.4%46.9%25.5%t(20)=3.2, p<.01

34. 34 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Results for simulator evaluation  Pretty consant training effect for all near transfer scenarios  Generalization only for particular far transfer scenarios

35. 35 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Results for simulator evaluation

36. 36 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Test of Hypothesis 2: Training effect of combined training program higher than for just PC-based training as in Pollatsek et al. (2006)? T-Intersection Amity-Lincoln Left Fork Opposing Truck Left Turn Pedestrians on Left Truck Crosswalk Adjacent Truck Left Turn Blind Drive Intersection with One-Way Street Mullins Center Bus Left Turn at Triangle St Signal Ahead at hill Right on Red Curved Stop Ahead Truck Blocking Travel in Lane Intersection with new green 11 scenarios included in evaluation! Results for simulator evaluation

37. 37 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Results for simulator evaluation The non-published variances and means for the RAPT-1 evaluation are based on former work at the Human Performance Lab and were provided for use in this study by Anuj Pradhan, graduate student at UMASS Amherst.

38. 38 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Results for simulator evaluation The non-published variances and means for the RAPT-1 evaluation are based on former work at the Human Performance Lab and were provided for use in this study by Anuj Pradhan, graduate student at UMASS Amherst.

39. 39 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Results for simulator evaluation SIMRAPT EvaluationRAPT Evaluation (Pollatsek et al. (2006) TrainedUntraineddifferenceTrainedUntraineddifference Near Transfer 67.536.131.448.427.221.2 Far Transfer 81.349.032.357.835.322.5 Total 72.242.329.952.031.120.9 The non-published variances and means for the RAPT-1 evaluation are based on former work at the Human Performance Lab and were provided for use in this study by Anuj Pradhan, graduate student at UMASS Amherst.

40. 40 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Outline  Review of Motivation  Experimental Design Training Experimental Group Training Control Group Simulator Evaluation  Scoring and Results Training Results HPL Simulator Results  Conclusion

41. 41 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Conclusions 3.RAPT/SIMRAPT  Ability to fixate on areas of potential risk on HPL driving simulator  Training succeeded for particular scenarios 2.SIMRAPT  Ability to recognize risks in driving task on Drive Square Simulator 4.Training effects exceed those of PC Training, though not significantly 1.RAPT  Recognize Risks from Plan View

42. 42 Master Thesis Defense, presented by Frank Diete, 09.27.2007  Generalization Suggestions for future research  Deversify training  Duration of the training  Redesign Training  Effects of simulator driving for control group  More extensive study necessary

43. 43 Master Thesis Defense, presented by Frank Diete, 09.27.2007 Thank you for your attention Additions? Suggestions? Any Questions?