1 SAVE-IT SAfety VEhicles using adaptive Interface Technology Phase 1 Research Program Quarterly Program Review Overview Gerald Witt & Harry Zhang August 12, 2003

2 SAVE-IT SAVE - IT Phase 1 Program Overview u Program Team u Mission and Objectives u Program Plan Summary u Technical Approach –Phase 1 Research model –Team coordination –Schedule –Human Factors research summary

3 SAVE-IT Program Team DDE Human Factors Team Leader Dr. Harry Zhang DDE Principal Investigator Dr. Harry Zhang Task Leaders Dr. David Eby Dr. Paul Green Dr. Bary Kantowitz Dr. Dave LeBlanc Research Focus Scenario ID Driving Task Demand Performance Telematics Demand Evaluations - on road UMTRI Principal Investigator Dr. David Eby U of Iowa Principal Investigator Dr. John Lee Task Leaders Dr. John Lee Dr. Dan McGehee Dr. Tim Brown Research Focus Distraction Mitigation Cognitive Distraction Telematics Demand Guidelines and Standards Evaluation Program Manager Gerry Witt Task Leaders Dr. Zhang Dr. Smith Research Focus Visual Distraction Intent Safety Warning Countermeasures Data fusion Benefits Analysis DDE Technology Team Leader Greg Scharenbroch Seeing Machines Inc. Ford Evaluation Principal Investigator Jeff Greenberg GM Evaluation Principal Investigator Scott Geisler A comprehensive program team has been assembled bringing a unique blend of expertise and complimentary capabilities. Task Leader Jeff Greenberg Research Focus Evaluation Task Members Tim Newman Dr. Branislav Kisacanin Nancy Edenborough Michelle Wilkes Development Focus Technology Development System Integration Eye Tracking Technology Task Leader Scott Geisler Research Focus Evaluation DDE Principal Investigator Greg Scharenbroch

4 SAVE-IT SAVE - IT Mission and Objectives Advance the deployment of adaptive interface technology countermeasures for distraction related crashes Enhance collision warning effectiveness by optimizing alarm onset based on driver’s workload or distraction Conduct comprehensive human factors research to derive distraction and workload measures for use adaptive interfaces. Develop and apply evaluation procedures for assessment of safety benefits Develop system operational performance requirements and guidelines for adaptive interface conventions Provide the public with documentation on human factors research findings for performance and standardization development Identify scalable system concepts and sensing technologies for further research to follow the SAVE-IT program O B C J E T I V E S Mission To demonstrate a viable proof of concept that is capable of reducing distraction related crashes and enhancing collision warning effectiveness Phase 1 Phase 2

5 SAVE-IT Program Plan Summary IOWAUMTRI DELPHI Phase I Research and Concept Development Phase II Data Fusion, System Integration and Evaluation Cognitive Distraction Visual Distraction Driving Task Demand Telematics Demand System Integration Vehicle build Demo. 13 Distraction Mitigation Identify countermeasures Cognitive distraction Visual distraction 4B Literature review Cognitive distraction Visual distraction 4A Safety Warning Countermeasures Identify demand levels 6B Validate demand levels 6C Literature review 6A Identify diagnostic measures 7B Develop and validate algorithms 7C Literature review 7A Identify diagnostic measures 2B Develop and validate algorithms 2C Literature review 2A Identify diagnostic measures 5B Develop and validate algorithms 5C Literature review 5A Performance Develop and validate algorithms 3C Identify diagnostic measures 3B Literature review 3A Intent Develop and validate algorithms 8C Identify diagnostic measures 8B Literature review 8A Scenario Identification Crash statistics analysis 1 Data Fusion 11A Distraction Mitigation 11B Safety Warning Countermeasures Subcontractors: 11 Validate countermeasures Cognitive distraction Visual distraction 4C Identify countermeasures 9B Literature review 9A Technology Development Technology / architecture concept identification 10A Technology / architecture concept car 10B Establish Guidelines & Standards 12 Program Summary and Benefit Evaluation 15 Iowa UMTRI Evaluation 14A Iowa 14B Ford 14C UMTRI 14D GM 14

6 SAVE-IT SAVE-IT Phase 1 Research Model Technology/Architecture Concept Development Adaptive Safety Warning And Distraction Mitigation System Architecture Concepts Scenario Identification Real Time Distraction Sensing Requirements Driving Performance Cognitive Distraction Visual Distraction Distraction Mitigation Telematics Demand Intent Safety Warning Countermeasures Countermeasure Technology Identification and HMI Concepts Driving Task Demand Distraction Assessment Data Fusion Concepts Situational Threat Assessment Concepts Concept Demonstration Human Factors Research Phase 2 Recommendations and development plan Diagnostic Research

7 SAVE-IT Preliminary SAVE-IT Model FW Long Range Target Assessment FW Short range R Long Range R. Short Range Environment HUD Eye Tracking/Oculometrics Bio Signs Haptic FB Phone Climate Phone MMM 3D Audio Y MMM IP Controls Brakes Throttle Steering Side Detect Throttle Brakes Steering Situational Threat Assessment Pedestrian Detect Vehicle Control Longitudinal Lateral Displays Flashers CHMSL HMI data fusion Processor Warnings Response Req’d Adaptive HMI Stimulate/Suppress Warning sensitivity Information priority N Driver State Monitor Provides real time assessment of driver distraction Provides global situational threat assessment Provides adaptive countermeasures. Provides a watchful eye when your not Increases safety guard band when required Substantially reduces perceived false alarm conditions and minimizes driver disregard.

8 SAVE-IT Team Coordination u Close team coordination is required to maintain consistency within research, experimental design and conclusions u A bi- weekly Human Factors team meeting is held via conference call. –Schedule review –Design Reviews –Issue discussions and resolutions –Commonization strategy discussion »Common dependant variables »Age groups, etc. u Common development process –Literature review »Report –Design/data collection »Team design reviews »IRB approval »Data collection »Findings and recommendations –Phase 2 planning »Algorithm development and validation plan »Preliminary Phase 2 research plan

9 SAVE-IT Phase 1 Schedule Milestones u Literature reviews complete, report submitted to NHTSA/Volpe9/10/2003 u Final Reports and Recommendations to Delphi12/31/2003 u Phase 2 planning documentation to Delphi12/31/2003 u Final report and recommendations to NHTSA/Volpe3/4/2004 u Phase 2 concept vehicle demonstration to NHTSA/Volpe3/4/2004 u Phase 2 planning documentation to NHTSA/Volpe3/4/2004

10 SAVE-IT SAVE-IT: Real-time Adaptive interface Common scenarios among tasks Driving & non-driving demands Driver state (distraction, intent, physiological measures) Safety warning systems Non-Adaptive- Interface Approach Arousal Approach Demand Approach Comprehensive Safety Management Systems CAMP Workload (U.S., Japan) De Waard GIDS (Europe) COMUNICAR (Europe) Comparison of Approaches

11 SAVE-IT Philosophy of Comprehensive Safety Management Systems u Driver impairment reduces overall attentional capacity. u Driver distraction increases the attention allocated to non-driving tasks and reduces the attention allocated to driving tasks. u Safe driving requires commensurate attention paid to driving tasks. u Required attention to driving tasks varies with driving task demand. u Objectives: To assess distraction, impairment, and driving task demand in order to ensure sufficient attention is paid to driving tasks.

12 SAVE-IT Task 1: Crash Statistics Analysis u Objectives –Identify crash scenarios (e.g., rear-end crashes) that the SAVE-IT program should be designed to prevent. u Major Findings –20-50% of crashes involve some form of driver distraction and inattention. –CDS appears to be best suited for the task. FARS, GES, and HSIS are not appropriate for this task. –Prior research indicated that single-vehicle-run-off-the-road and rear-end crashes are most common scenarios in which driver distraction is a causative factor. –Prior research indicated distracting events include interior (e.g., radio, cell phones, passengers) and exterior (e.g., scenery) objects and events. u Current Status (55% completed) –Literature review report (to Delphi)Completed –Crash data analysis planIn progress –Crash data analysisSept.-Nov. ’03 –Expert panel meetingNov. ‘03

13 SAVE-IT Task 2: Driving Task Demand u Objectives –Determine the level of attentional demand imposed by the driving environment that represents the required level of attention allocated to the driving tasks. u Major Findings –Analysis of crash data is key because crash rates can be assumed to indicate environmental unpredictability and the amount of attention demanded by the environment. –HSIS is the best suited crash database because it contains information about environmental conditions (e.g., weather, traffic volume, road surface conditions) at the time of crashes. –In laboratories, driving demand can be approximated by visual demand (% of time needed to look at the road to drive safely) as measured with the visual occlusion method. u Current Status (2A 50% completed; 2B 60% completed) –Literature review report (to Delphi) (2A)Completed –HSIS database preparation, review of analysis plan (2A)Completed –Crash data analysis (2A)Aug.-Sept. ’03 –Review of test plan, simulator preparation, pilot testing (2B)Completed –“Visual occlusion” simulator experiment (data collection) (2B)In progress –Data analysis and algorithm development (2B)Sept-Nov. ’03

14 SAVE-IT Task 3: Performance u Objectives –Determine performance measures/variables that are diagnostic of driver distraction. u Major Findings –Literature review indicated that there exists very limited data (e.g., distribution data, eye glance data) comparing driving with and without various in-vehicle devices (e.g., radio, phone, navigation device). –NHTSA’s 100-car naturalistic driving study currently conducted at Virginia Tech should be very useful. –Normative data on drugs and driving can be useful. u Current Status (3A 15% completed; 3B 55% completed) –Identification of research needs/gaps (3A)Completed –Literature review report (to Delphi) (3A)In progress –Instrumented car preparation, pilot testing (3B)Completed –Review of test plan (3B)Completed –On-road experiment (data collection) (3B)Summer ’03 –Data analysis and algorithm development (3B)Sept-Dec. ’03

15 SAVE-IT Task 4: Distraction Mitigation u Objectives –Develop appropriate countermeasures that can mitigate against excessive levels of distraction. u Major Findings –Research on computer etiquette, negotiated access, and automation challenges is useful. –Used four focus groups (24 participants at Iowa City & Seattle) to determine what activities drivers find distracting (driver/system initiated, technology/non-technology oriented) and what mitigation strategies they prefer. Potential technology for mitigating distraction is viewed positively by some and negatively by others. –Potential countermeasures such as warning, informing, advising, demand minimizing, prioritizing/filtering, locking, etc. can be summarized by a model-based taxonomy of mitigation strategies with degree of intervention and locus of control (driver vs. IVIS) as the dimensions. u Current Status (45% completed) –Literature review report (to Delphi) (4A)In progress –Focus group study (data collection)(4B)Completed –Focus group data analysis and draft report (4B)Completed –Revision of mitigation taxonomy based on focus group input (4B)Completed –Cognitive task analysis (4B)Aug.-Dec. ‘03 –“Driver acceptance” simulator experiment (4B)Aug.-Dec. ‘03

16 SAVE-IT Task 5: Cognitive Distraction u Objectives –Determine which measures (performance, driver state, and vehicle state variables) are diagnostic of cognitive distraction. –Develop an algorithm relating diagnostic measures to performance (including RT). u Major Findings –Cognitive distraction may be manifested in terms of driver state (eye movements, scan patterns, ocular responses, psycho-physiological measures), driving performance, and vehicle system state. –Theories and models such as multiple resource theory, malleable resource theory, strategic task management (switching), and ACT-R can be useful. –Hidden Markov Models (representing stochastic sequences where states are not directly observed but are associated with a probability density function) and Support Vector Machines (determining optimal hyperplane separating two classes) will be very useful in predicting driver distraction. u Current Status (40% completed) –Literature review report (to Delphi) (5A)In progress –Experimental design (5B)In progress –Simulator experiment (5B)Aug.-Dec. ‘03

17 SAVE-IT Task 6: Telematics Demand u Objectives –Determine distraction potential and prioritization for commonly-used telematics functions. u Major Findings –Distraction potential may be measured in terms of task completion time, number of errors, number of glances, mean glance duration, reaction time, etc. –Current guidelines (e.g., Alliance’s Statement of Principles, SAE J2364) and IVIS Demand model for key task characteristics (visual, auditory, cognitive, manual) can be very useful. u Current Status (6A 60% completed, 6B 2% completed) –Summary of prior research (6A)Nearly completed –Literature review report (to Delphi) (6A)In progress –Preliminary test plan (6B)Completed –Simulator experiment (6B)Sept.-Dec. ‘03

18 SAVE-IT Task 7: Visual Distraction u Objectives –Identify eye glance measures that are diagnostic of visual distraction and that can be used in real-time, adaptive interface technology systems. –Determine performance (including RT) effects of visual distraction. For example, RT = f(glance duration, glance frequency, etc.). u Major Findings –Prior research indicated that visual distraction (off-road glances) degrades driving performance (e.g., SDLP, lane departures, RT) and increases the likelihood of crashes. –Prior research rarely used automatic eye tracking systems to measure visual behaviors in real time and focused on task-based (e.g., radio tuning) rather than time-based visual behaviors. –Real-time measurement of time-based visual behaviors is critical to SAVE-IT. –Current experiments measured visual behaviors in real time using Seeing Machines eye tracking system and indicated that visual distraction degraded performance (e.g., RT). u Current Status (7A completed; 7B 40% completed) –Literature review report (to Delphi) (7A)Completed –Experiment 1 data collection (7B)Completed –Review of Experiment 2 test plan, facility preparation (7B)Completed –Experiment 2 data collection (7B)In Progress –Data analysis, algorithm development (7B)Sept-Dec. ’03

19 SAVE-IT Task 8: Intent u Objectives –Identify a list of Intents that are measurable and potentially useful for distraction mitigation and safety warning countermeasures. –Determine diagnostic measures for reliable detection of those intents. u Major Findings –Information about driver intent can improve system effectiveness and acceptance and reduce nuisance alerts. For example, FCW warnings may be delayed/suppressed if drivers intend to brake, and blind spot warnings may be issued when drivers intend to change lane. –Intent detection variables can be classified into affordance (e.g., exit ramp), motive (e.g., navigation info), kinematics (e.g., raw, heading), control (e.g., turn signal), and eye glances. u Current Status (50% completed) –Literature review report (to Delphi) (8A)Completed –Framework for intent determination (8B)Completed –Acquisition and preliminary examination of ACAS FOT pilot data (8B)Completed –Naturalistic Lane Change Data »Markov matrix analysis of eye movements (8B)Completed »Analysis of kinematic & control variables (8B)In progress

20 SAVE-IT Task 9: Safety Warning Countermeasures u Objectives –Improve the effectiveness and acceptability of safety warning systems by designing these systems to adaptively respond to intent, distraction, and demand information. u Major Findings –Although adaptive systems can fail because of poor user acceptance (e.g., not understanding the system, perceived system inconsistency or unpredictability, drivers feeling no longer in control) and poor design (e.g., oscillations), they have shown some promise and can be acceptable to drivers. –Excessive nuisance alerts pose major problems for FCW and other warning systems. Nuisance alerts may be reduced by adjusting warning criteria using driver state information (distraction and intent). »FCW warnings and lane drift warnings may be delayed/suppressed if drivers are attentive or intending to engage in particular maneuvers. »Blind spot warnings may be issued when drivers intend to change lane. u Current Status (45% completed) –Literature review report (to Delphi) (9A)Completed –Identification and definition of adaptive enhancement issues (9B)Completed –Selection of FCW algorithm and DVI (9B)Completed –Preliminary design of experiments (9B)Completed –2 simulator experiments (9B)Sept.-Dec. ‘03

21 SAVE-IT Common Issues and Solutions u Regular team meetings convened to –Discuss common issues and solutions. –Commonize design and variables across experiments as much as possible. u Dependent Variables –Four common dependent variables used across all experiments »Reaction time for initial brake application »Reaction time for foot off accelerator »Steering entropy »Steering reaction time and direction –Many other variables (e.g., SDLP, lane departures, head and eye movements) will be used, although they may vary in different experiments. u Subject Ages –Subject ages divided into 3 age groups: 18-25; 35-55; and years old. –All experiments must include the group of years old. –Due to time and budgetary constraints, there is no requirement to use the younger (18-25 years old) or older subjects (65-75 years old). If multiple age groups are used, however, the additional group(s) may be years old, or years old, or both. –The choice of subject ages is made to balance the age effect and age representation (e.g., inclusion of subjects of 45 years of age or older) years old are also the likely initial adopters of SAVE-IT technologies.

22 SAVE-IT Common Issues and Solutions u All simulator experiments (at Delphi, Iowa, and UMTRI) use GlobalSim Simulator. u All visual glance behaviors measured with Seeing Machines eye tracking system. u If applicable, the driving scenario is the “lead vehicle following” scenario. –The same vehicle (white passenger car) used as the lead vehicle in all experiments. –For all experiments, a “rubber-band” control (Lee et al., 2002) is used to set time headway at 1.8 s at the moment of lead vehicle braking. –Except for Task 9, lead vehicle braking is non-imminent (e.g., -0.2 g). –Lead vehicle braking is unpredictable and infrequent (e.g., less than 1 per minute). u If practical, both rural roads and freeways are used. –Target speed = 45 mph for rural roads and 65 mph for freeways. –Some experiments use one type of roads only because the use of both types of roads may result in excessive long sessions or the use of too many subjects. u Per Volpe’s request, the Hidden Markov Model will be considered as a method in Tasks 3 (performance), 5 (cognitive distraction), and 8 (intent). u The minimum number of subjects per condition is 8. Many more subjects are used in many experiments.