1. Task-Centered Performance Modeling Analysis Swapna Ghanekar Supervisor: Dr. Fatma Mili Research was sponsored by TARDEC

2. Introduction Systems in which there is a human in the loop are hard to design, analyze, and verify. When the correct behavior of these systems is safety-critical or mission-critical, considerable effort must be invested in the design of the human-machine interface and in the analysis of the integrated human-machine behavior. Although accidents are often attributed to “operator error”, these errors can often be predicted and avoided by accounting more thoroughly for the human in the loop [6]. Decisions concerning the human-machine interface are more than afterthought cosmetic decisions of layout, color, and font; they concern the essence of the functionality of the system encompassing decisions concerning which tasks to automate and which tasks to leave for the human operator and, for those tasks that are not fully automated, deciding for the support role that the system can play (see, e.g. [3]). These collaborative decisions are made taking into account the capabilities and limitations of the human agent

3. Introduction, cont. including the speed of making complex decisions, the capacity to recall information, the accuracy with which this information is recalled, as a function of its recency – among other factors, the effect of mental overload, and the effects of other human behavioral modifiers such as fatigue, stress, fear, and other environmental and emotional conditions [1, 2, 7-10]. The detailed qualitative and quantitative analysis of the functionality and service delivery of systems with a human in the loop are an important component of design. Performance modeling [4, 5] and cognitive modeling [1, 2] are the main approaches available to perform such analysis. The models generated are, by nature, very detailed (tasks described at the mouse click level), and thus labor intensive. Furthermore, because models tightly integrate elements of the service at hand (e.g. select a new itinerary) with elements of the interface (point and click on an itinerary from a menu) and

4. Introduction, cont. characteristics of an agent performing a task, they make it very difficult to reuse a model if any one of these factors changes. The lack of reuse proved to be a major impediment to the widespread use of human- centered system analysis during design.

5. Motivation System design must account for the role of human operators. Relevant questions include: What tasks should be assigned to operators? What combinations of tasks should be assigned to the same operator? What tasks are better automated? What support can the system provide to human operators? The main goals of this project are ensuring the reusability of modeling information and providing early insight into tasks, task assignments, and system design.

6. Approach The approach is to develop a repository, or ontology, of tasks. This repository must contain all information inherent to the task relative to performance and may not contain any information specific to agents performing the task. The ontology becomes an evolving reflection of our understanding of tasks and can be used to develop alternate designs, reason about them, and compare them.

7. Approach, cont. Task modeling Performance & Cognitive Modeling Design decisions Task scheduling & assignment Task analysisAgent Modeling Resource modeling Figure 1: Modeling Approach

8. Approach, cont. The ontology is used for task, agent, and resource modeling from Figure 1. This gives insight into task analysis, task scheduling and assignment, and design decisions. Ultimately, this aids in performance and cognitive modeling. The idea behind this approach is to reuse as much task knowledge as possible. We use an ontology because it is a good vehicle for capturing, sharing, and reasoning about knowledge. We used Protegé /OWL-S to develop the ontology as shown in Figure 2.

9. Approach, cont. Figure 2: Protegé/OWL-S

10. Ontology of Tasks In order to build an ontology of tasks, we need the following information: Structural information The components of the task The relationships between components Resources required to perform the task Nature, Quantity State transformation performed by the task Precondition Effect Data transformation performed by the task Input Output

11. Structural Information The components of a task can be modeled as atomic, simple, or composite processes. An atomic process is the most basic process and cannot be decomposed. A simple process can be realized by an atomic process, can stand alone waiting for refinement, or can expand to a composite process. A composite process collapses to a simple process and is a refined view of that simple process. Figure 3 shows a simple process that expands to a composite process in Protegé.

12. Structural Information, cont. Figure 3: SimpleProcess expandsTo CompositeProcess

13. Structural Information, cont. Figure 4: CompositeProcess composedOf ControlConstruct

14. Structural Information, cont. A composite process is composed of a control construct (split, sequence, etc.) as shown in Figure 4. The control construct defines the relationships between processes and governs the execution order of the processes within a composite process as shown in Table 1. Control ConstructOrder Any-OrderBag of processes If-Then-ElseChoice of one process or another SequenceTime-ordered list of processes SplitBag of concurrent processes Table 1: Control Constructs and Execution Order

15. Required Resources All processes use resources that have a quantity and type (cognitive, perception, and motor). Additionally, simple processes and composite processes also have computed resource usages. Figure 5 shows a composite process with resources and the details of a particular resource. The computed resources are based on the control constructs and are calculated for each of the three resource types. Resources are summed for Any-Order, Sequence, and Split. The maximum resource value between the then-process and the else-process is used for If- Then-Else.

16. Required Resources, cont. Figure 5: Composite Process and Resources

17. State and Data Transformation Processes may have inputs, outputs, preconditions, and effects. Figure 6 shows an atomic process with input, output, precondition, and effect along with cognitive, perception, and motor resources. Figure 6: Modeling Approach

18. Transformation, cont. Composite processes should not have conflicting preconditions and effects. The sequence in Figure 7 has a conflict because Atomic Process 1 has an effect that x=7 and is followed by Atomic Process 2 that has a precondition that x=2. Figure 7: Processes That Conflict

19. Transformation, cont. The sequence in Figure 8 does not have a conflict because all preconditions are satisfied. Figure 8: Processes That Do Not Conflict

20. Transformation, cont. Tasks can be combined as shown in Figure 9. The final result is the overall precondition and effect for the group of atomic processes. Figure 9: Combining Processes

21. Reasoning An ontology supports reasoning about tasks by: Propagating changes: Changes must be propagated through the ontology. If the resource usage of a process, P, changes; then, the computed resource usages of all processes that have P as a sub-process must change. Ensuring that data is consistent: Preconditions and effects may not be in conflict.

22. Reasoning, cont. Optimizing task structure: The ontology should recognize if multiple tasks do not require the same resources and can be done concurrently by one agent. For example, a person can speak on the phone and look through a help manual. Analyzing different task-agent assignment: The ontology should recognize how to assign multiple agents to a set of tasks.