1. Sandy Lozito ATM2003 June 2003 The Impact of Voice, Data Link, and Mixed Modality Environments on Flight Deck Procedures Sandy Lozito 1, Savvy Verma 2, Lynne Martin 2, Melisa Dunbar 2 and Alison McGann 2 1 NASA Ames Research Center, CA, USA. 2 San Jose State University, CA, USA June 23 - 27, 2003

2. Sandy Lozito ATM2003 June 2003 Why Study Mixed Modality? n Introduction of controller pilot data link communications is intended to help solve the frequency congestion problem, however voice communications will continue  Implementation of data link is clearance-specific  Data link and voice will co-exist  Voice amendments to data link messages due to negotiations/ temporal constraints

3. Sandy Lozito ATM2003 June 2003 Voice and Data Link Procedural Constraints Temporal quality of voice may lead to more immediate response (Sorkin, 1987) Permanence of data link (review menu) leads to flexibility in information access Data link in its current implementation has sequential constraints Voice allows for simultaneous data entry and response

4. Sandy Lozito ATM2003 June 2003 Previous Research Data link and mixed modality research  Data link research has shown an increase in accuracy and consistency, but at the cost of lengthening time for information transfer (Kerns, 1989; Lozito, McGann, & Corker, 1993)  Tallotta et. al (1988) examined data link as a redundant medium for voice and found this usage increased ATC workload  McGann, Morrow, Rodvold, Mackintosh (1998) found that when introducing time pressure into a mixed modality environment, it may negatively impact both data link and voice

5. Sandy Lozito ATM2003 June 2003 Mixed Modality and Time Pressure To reexamine issues involved in shifting modalities  Hypotheses  Voice communication would suffer more than data link in mixed environment  Closely spaced messages Will have longer acknowledgement times Will result in more errors and clarifications

6. Sandy Lozito ATM2003 June 2003 Current Research Mixed Modality and Time Pressure Procedure Flight deck perspective Participants  5 B-747-400 type certified flight crews  Average flight hours 11,100 hrs  Average flight hours in B-747-400 = 987 hrs Procedure (within subject design)  6 enroute scenarios (2-voice, 2-datalink, 2-mixed)  4 pairs of target messages in each scenario  Mixed environment (voice always followed data link)

7. Sandy Lozito ATM2003 June 2003 Design Independent Variables  Modality (voice only, data link only, mixed)  Order (in mixed voice always followed data link)  Interval (duration between the two messages)  5 seconds or 60 seconds

8. Sandy Lozito ATM2003 June 2003 Design Dependent variables  Acknowledgement Time  Voice: Onset of ATC message to the end of pilot read back including any clarifications  Data Link: Message arrival at the simulator (Initial alert to crew) to acknowledgment via data link  Errors  Inaccurate read back or read aloud  Data entry errors  Clarifications  Questions related to message content

9. Sandy Lozito ATM2003 June 2003 Design Data Link Implementation and Procedures Data link implemented on forward FMC/CDU  Textual  Time-shared Pilot-Flying and Pilot-Not-Flying tasks maintained Both pilots asked to read message prior to response Pilots asked to assign higher priority to voice clearances Pilots asked to respond to the messages in the same modality

10. Sandy Lozito ATM2003 June 2003 FMC/CDU Display of a Data Link Message

11. Sandy Lozito ATM2003 June 2003 Four Pairs of Messages in Every Scenario Six Enroute Scenarios Flown (2 voice, 2 data link, 2 mixed) Message 1a1 minute after readback/accessMessage 1b (Example) Descend to and Maintain FL370Cleared direct ILC (Wilson Creek) Reduce speed to mach.84Descend and maintain FL350 Message 2a5 seconds after readback/accessMessage 2b Message 3a1 minute after readback/accessMessage 3b Message 4a5 seconds after readback/accessMessage 4b

12. Sandy Lozito ATM2003 June 2003 Results - Acknowledgment Times Single Modality- Data link or Voice Order and interval have no effect on acknowledgment times for single modality-voice Order and interval interaction on acknowledgement times for single modality-data link  Short interval between messages increased acknowledgment times for the 2 nd message compared to other acknowledgment times within data link

13. Sandy Lozito ATM2003 June 2003 Results - Single-Modality Acknowledgment Times Mean total transaction times (acknowledgment times) by modality, interval, and order with +/-1 SEM bars 0 5 10 15 20 25 30 35 40 45 50 VoiceData Link 1st Long 2nd Long 1st Short 2nd Short Mean total transaction time in seconds

14. Sandy Lozito ATM2003 June 2003 Results - Acknowledgment Times Mixed Modality - Data Link and Voice Data link is always the first message Significant main effects (p <.05) for order and interval  Longer acknowledgment times for 1 st message (data link) compared to 2 nd message (voice)  Short interval between message increased acknowledgment times compared to long interval for both voice and data link

15. Sandy Lozito ATM2003 June 2003 Results - Mixed-Modality Acknowledgment Times Mean total transaction times (acknowledgment times) for mixed modality by interval and order with +/-1 SEM bars 0 10 20 30 40 50 60 Mixed modality 1st Long 2nd Long 1st Short 2nd Short Mean total transaction time in seconds DL VV

16. Sandy Lozito ATM2003 June 2003 Matching Messages Between Single and Mixed Modality - Example All Voice All Data Link Mixed Modality Message 1A Long Interval Message 1B Long Interval Message 1A Message 1B - V Long Interval Message 1A-DL Message 2A Short interval Message 2B Message 2A Short Interval Message 2B Message 2A -DL Short Interval Message 2B-V

17. Sandy Lozito ATM2003 June 2003 Results - Acknowledgment Times Single v. Mixed Environment For voice, the 2 nd message in the single modality- voice was matched to the 2 nd message in mixed environment  Significant interaction between interval and environment for acknowledgment times  Voice messages following a short interval in the mixed environment have longer acknowledgment times than the other three message types (single environment voice, voice in mixed with long interval) For data link, the 1 st message in the single modality-data link was compared with 1 st message in mixed modality  Significant main effect for interval on acknowledgment times  Acknowledgment times longer in short interval

18. Sandy Lozito ATM2003 June 2003 Results - Acknowledgment Times Mean total transaction times (acknowledgment times) by modality, interval, and order modality Order & Interval

19. Sandy Lozito ATM2003 June 2003 Results - Clarifications Review Log and Voice Short interval significantly increased the number of clarifications

20. Sandy Lozito ATM2003 June 2003 Results-Voice Clarifications Only Examined type of voice clarifications as an indication of the cognitive processes  Mixed environment (12/17 of voice clarifications)  6 out of 12 requested repeat of full clearance  6 out of 12 requested clearance confirmation  Single environment - voice (5/17 of voice clarifications)  4 out of 5 requested clearance confirmation  1 out of 5 requested repeat of full clearance

21. Sandy Lozito ATM2003 June 2003 Results - Errors Four messages were not acknowledged  All voice  All in the mixed modality environment Seven messages for which the crews failed to implement the entire clearance (e.g., did not implement a speed change)  All seven occurred with messages separated by a short interval  Six in the mixed environment (one in data link)  Five were voice clearance elements

22. Sandy Lozito ATM2003 June 2003 Discussion Time pressure differentially affected voice and data link  Single modality environment  In single modality-data link, time pressure lengthened data link acknowledgment times for the 2 nd message  In single modality-voice, time pressure had no effect on acknowledgment times  Voice offers more flexibility  Current data link implementation and procedures may not be amenable to time pressure

23. Sandy Lozito ATM2003 June 2003 Discussion Time pressure and the mixed modality environment  Acknowledgment times lengthened for each modality in the mixed environment when time pressure existed  In mixed modality environment, crews seem to interrupt the handling of a data link message to handle a voice message when time pressure exists  Instructions to assign voice a higher priority may have impacted this result

24. Sandy Lozito ATM2003 June 2003 Discussion Time pressure and the mixed modality environment  There may be incompatibilities between voice and data link procedures that result in difficulties transitioning between the two when there is time pressure  May increase acknowledgment times for both  May increase confusion regarding message content  May increase errors in following message instructions Inconsistencies within the review log  Lack of accuracy in the review log due to voice amendments

25. Sandy Lozito ATM2003 June 2003 Future Directions Further examination of mixed modality and crew procedures using human performance modeling Investigate changes in procedure or interface that might mitigate the impact of mixed modality with time pressure  Air MIDAS (Corker & Smith, 1993)  Add data link procedural representation within the model  Add data link functions and tasks