Analysis of Driving simulator validation for speed research Stuart T. Godley, Thomas J. Triggs, Brian N. Fildes

1.Purpose of Research ideas/question  To evaluate countermeasures for mean speed using behavioral validation of an advanced driving simulator  Using behavioral validation : relative validity, absolute validity and interactive validity:  to validate the Monash University Accident Research Centre (MUARC) driving simulator.

2. Background support  Blaauw(1982) proposed two levels of validity:  Physical validity is correspondence of simulator components, layout, dynamics with real car  Behavioral Validity is correspondence between simulator and car in the way the human operator behaves  Rel val. is necessary unlike absolute

Background support: Absolute Vs Relative Validity Background support: Absolute Vs Relative Validity

3. Theoretical Basis/ Hypothesis  Analysis of behavioral validation, absolute validity and relative validity is the theoretical basis.  Beh. Val. :comparison between simulator and real car driving  Absolute Validity: comparison of performance difference between experimental conditions in sim. and car  Relative Validity: difference between experimental conditions of identical magnitude

4. Practical Contribution  Simulator provides experimental control, efficiency, expenses, safety, ease of data collection which may/may not be provided by real car  Driving patterns in the simulator observed to improve future cars

5. Theoretical Contribution  For the purpose of effective countermeasure, speed is validated to be dependent variable for research using simulator.  Validity of speed covers research investigating the relative differences between treatment and control roads.

6. Appropriate Methodologies  Driver’s speed response to rumble strips was experimental stimuli, simulator adjusted  Two experiments conducted; driving instrumented car and simulator  Variables  Instrumented car : 12 males, 12 females, average age 29.8 years  Three sites; approach to stop sign, right curve, left curve  2 Treatment sites for simulator and real road: Consisted of rumble strips  2 control sites for simulator and real road: without rumble strips  Treatment and control matched for on road off road details

Methodologies...  Rumble strips : width 60 cm, preceded intersection by m, spaced at decreasing distances  Two experimental routes: half male/female on each route, had vehicles parked on them Method  Instructed to drive normally, experimenter in front seat, technician at back  First a practice drive:11 min, then experimental drive: ~40-50 min. Data collection Instantaneous speed recorded at each intersection

Methodologies...  Some participants excluded from data pool for individual treatment-control analysis: unrestricted drive to intersection, slow moving car, school children walking, etc  No participant removed from all analysis Validity of measures  Participants had to sign a consent form approved by Monash University Ethics Committee

Methodologies... 2 nd method: Simulator Experiment Variables  Simulator car : 12 males, 8 females, average age 26.4 years  Projected through computer :resolution 640x 480 pixels, rate of 30 Hz. 180 deg forward, 60 deg horizontal, 45 deg vertical view  Rest all variables are same Method and data collection were the same; except there was no hindrance while driving; hence collection of data

7. Statistical Analysis  A modified correlation analysis based on canonical correlation was used.  Correlation of each participant’s data not feasible: separate participants used in each exp  Measures of effect size in ANOVA are measures of degree of association between effect and dependent variable (ex: main effect, linear contrast, an interaction)  Two one-way ANOVA calculated: one for treatment, other for control  Omega squared and interclass correlation estimate degree of association in presentation

Statistical Analysis...  Non significant results validate simulator, but may arise from inadequate statistical power rather than genuine absence of difference.  Small omega square reflects genuine non- differences; large value suggests a difference exists, but insufficient sample size.  Omega square at or below 0.01 meaningless

F- test used to interpret the significant differences in experiments Alpha = % confident that differences are significant Correlation used to support interactive relative validity Size effect verify if the non-significant result can be conclusive 8.Presentation of Results

Results : Stop Sign Approach

 Significant difference between mean speed at treatment site relative to the control site ( P<0.001 )  Very small size effect (0.002) to support this non-significant result  Average relative validity was established  The pattern of speeds for the treatment site and control site similar in both experiments, supported by a significant correlation (0.40)  Interactive relative validity established  Speeds for the two sites converged as they approach the intersection in the simulator  This pattern was not observed in the data for the on-road experiment  Absolute validity was not established

Results:Right Curve Approach Approach

 Car: No significant difference between the mean speed  Simulator: Significant difference between the mean speed  Significant interaction between 2 experiments (P<0.01)  Average relative validity was not established  Speed pattern similar for the first three-quarters of the measurement area  Significant correlation (0.52) supported interactive relative validity  Right curve treatment site speeds not statistically different between the car and simulator experiments (P = 0.590)  Small effect size(0.007) supports this non-significant result.  Control site speed slower for instrumented car experiment compared to simulator experiment (P<0.05)  Absolute validity established for treatment sites, but not control sites

Results:Left Curve Approach Approach

 Significant difference between mean speed at treatment site relative to the control site (P<0.001)  Significant speed differences between the two experiments (P<0.001)  Averaged relative validity not established  Same speed pattern for the first three quarters of data collection area  Significant correlation (0.50) supported interactive relative validity  Significant speed differences in treatment site for instrumented car and simulator experiments (P<0.001)  Control site speeds not significantly different, but produced a size effect close to medium  Non-significant result cannot be considered as a conclusive.  Absolute validity was not established.

Relative validation:   Treatment speeds started to reduce relative to their control sites, before the rumble strips   Significant correlations in relation to interactive relative validity.   Rumble strips led to an average slower approach speed to the stop sign intersection in both the simulator and on-road trials   Relative validity well established for mean speed Absolute Validation:  Absolute speed values were generally different in the two experiments  General trend: simulator induce slower speeds than instrumented cars Results: Relative and Absolute Validation

9. Conclusion : Relative Validity  Simulator produced larger average speed differences between its treatment and control sites during the curve approaches  Two reasons: 1.On-Road : Speeds initially faster at the treatment relative to the control sites ; Non Perfect Site Matching  Wider lane widths conducive to faster driving  Treatment site roads may have geographical characteristic that encouraged faster speeds 2. Treatment site speeds started to reduce further back before the first of the rumble strips in the simulator experiment  main difference between the simulator and the on-road experiments  Without the two experimental differences, average relative validation for the curves may have been established.

 Absence of G-force motion cues in simulator  Intentionally short practice section may have contributed to simulator participants to slow earlier when they perceive the rumble strips  Converging speed pattern could only be determined when the vehicle stopped for on-road trial, rather than when the front of the simulator car crossed the stop line  Simulator may have slightly enhanced the speed differences  Address limitations and these differences should be able to be minimized.  Simulator does seem to be a valid tool for generating and generalizing relative speed results for experiments involving road based speeding countermeasures aiming to influence decelerating. 9. Conclusion : Relative Validity

9. Conclusion : Absolute Validity  Different absolute speed values in two experiments not particularly negative because simulator experiments only concerned speed differences between road environments  Experiments did not attempt to establish the numerical speeds at which the investigated road manipulations encourage drivers to drive

9. Conclusion :Overall  Evidence to conclude that speed is a valid measure to use for the experiments on driving simulator involving road based speeding countermeasures.  Speed profiles found indicate a speed reduction relative to control roads or other roads.  Speed has been clearly validated as a dependent variable for research using the simulator.  Validity of speed only covers research investigating the relative differences between road treatments and control roads.  Inconsistencies between the two experiments account for:  1. Differences in characteristics of the road and road environment between the treatment and control sites in the on-road experiments hindered the validation.  2. Procedural methods with the simulator, notably practice, may have also contributed.

10. Future Work/Research Directions:  Reduce the inconsistencies between the two experiments.  Include investigating absolute numerical speed values for validity of speed  More similar initial speeds for the on-road curve sites, so that a more similar treatment-control site speed difference to the simulator experience may be found.  Conduct experiment with same participants for the on-road and simulator trials so as to observe any difference in the results.  Experiments can be modified so that converging patterns can appear for both on-road and simulator data.  Attempt to establish the numerical speeds at which the investigated road manipulations encourage drivers to drive  Validity of speed to investigate the absolute differences between road treatments and control roads.