Day 2 Prior to today Participated in the one day class on Cognitive Ergonomics, read all material of that class studied the reader for today read the paper by vdVeer & Mariani

After today management plan and time schedule interviews with potential users video records for interaction analysis review relevant information from web and other document sources version 1 of Task Model 1 collect ideas for task model 2 prepare presentations for next time: management plan task analysis methods and techniques used: why, how, issues task model 1

5. Task analysis of complex systems Overview 5.1. Task analysis in the process of designing complex systems 5.2. Describing current work situations: Analytic methods Understanding implicit expert knowledge Explicit group knowledge (video illustration- Boeing accident) Understanding implicit group knowledge (video ill. Examination system) 5.3. GTA 5.4. Envisioning the future system 5.5. Evaluating task model 2

5.1. Task analysis in the process of designing complex systems Design of complex systems for users: the virtual system and the mental model Design for the mental model: 3 types of design activities analysis specification evaluation “Task analysis”, in modern approaches, indicates analysis of the whole task domain: business, work, users, stakeholders, situation “Specification” focuses on details of technology

work organization/ practice Client users’ knowledge/ behavior/needs Technology Task Model 1 Task Model 2 Scenario Simulation Prototype Functionality Dialog Representation Implementation usability measuring ethnography psychological knowledge acquisition/ hermeneutics problem analysis/ specification specification/ negotiation constraints/ opportunities feedback specification early evaluation early evaluation UVM maintaining consistency Documents/ artifacts validity analysis As soon as the system is implemented

Task analysis –describing the current situation task model 1 –envisioning the future task model 2

Task analysis in literature Several meanings collecting knowledge about, and describing, current task domain (task model 1) specifying new task situation, envisioning use of novel technology (task model 2) specifying details of technology (existing or new) (the user virtual machine) approaches always consider type of task knowledge, and relate this to model intended for design process Literature and approaches often consider only one or two, and often mix up meaning of what is being modeled

Important concepts and distinctions task and goal: each task (activity) has a goal (state to reach) a goal may be reached through several different tasks primary and secondary tasks a primary task is generic for the task domain irrespective of technology and situation a secondary task is related to the use of tools and technology basic and unit tasks a users’ unit task is an atomic primary task a basic task is the simplest task available for delegation to technology actions an action has no goal, its meaning is derived from the task it is part of

5.2. Describing current work situations: task model 1 Brigitte Jordan (1996): sources of task knowledge Explicit expert knowledge facts of task domain heuristics skills Explicit group knowledge process models, rules instructions historic documents anecdotes Implicit expert knowledge “expertise” intuition common sense Implicit group knowledge “culture” COP (community of practise) work practice

Analytic methods: conceptual framework set of basic concepts (task, goal, condition, action) relations between concepts formal representations, based on conceptual framework task hierarchies (HTA, based on Taylor) task-action grammars (TAG, ETAG), goals and operation rules (GOMS, CCT) planned process steps in collecting knowledge, modeling, specification focus on the individual human and his knowledge cognitive psychology

TKS (task knowledge structure) P. Johnson & H. Johnson (QMW college, London) conceptual framework tasks, plans (task structures), strategies procedures (object - action relations) roles (sets of tasks assigned to certain individuals) Psychological basis is conceptual knowledge in long term memory psychological techniques: interview, observation, card sorting document analysis

TKS (continued) method: from individual TKSs to generic task model GTM (task model 1) from GTM to Specific task model STM (task model 2) from STM to spec. interface model SIM (functionality at conceptual level) problems: experts often do not know details of others’ roles often there is no overview of all roles in an organisation objects (things) are only analysed at secondary level

ATOM - Paul Walsh (STC Technology, Harlow, UK) Start of analysis is identification of “principal objects” active objects (agents) passive objects (things) environment (includes things, agents, technology objects structured in class hierarchies and relations between types actions establish a causal link between object states actions are identified secondary, identified by type of verb: process verbs (d.g. “dies”) state verb (“waits”) action verb (“transports to”)

MAD - D.Scapin, S. Sebillotte (INRIA, France) Conceptual framework tasks, goals of people - states to be reached functions of systems (delegated subtasks) task hierarchy, subtasks, elementary tasks constructors (PAR, ALT, PAR, LOOP, COND, OPT) initial state, final state (describe change due to effect of task) prior condition, post condition Formalism task / goal tree (HTA)

MAD (continued) The Graesser method: describe as complete and detailed as possible how you …. (name the global label of the task to be described) “... because...” “... Therefore...” keep track of hierarchical structure (need a second analyst) filling the gaps “How ?” “Why ?”

MAD (continued) Generic levels of task knowledge abstract general task formulation specific context sub-tasks (expert task language) domain independent procedures: (“store a copy”, “send a message”) elementary actions (“ask for signature”, “put date on record”)

Understanding implicit expert knowledge Implicit knowledge, behavior, skills, “expertise” individual knowledge that can not be verbalized Technique: observation, registration Hermeneutic interpretation “what would be my knowledge and mental process, if I would show the behavior that this expert is showing” Reliability problem needs training and multiple interpreters

Explicit group knowledge Explicit sources: official –documented rules –documented business plans –documented laws, regulations –memos in-official –artifacts (incl. documents) –stories, anecdotes (location dependent) Validity: example: rules, regulations, laws –intention –date of installation –friction with novel developments –"officially still valid" –overruled

Example: analysing history documents (Video “Kegworth fatal error” - training video HRA) An example: the Kegworth accident Failure of a safety critical system: a Boeing crash at East Midlands Hardware failure: left motor flutter up-scaling problem no operational test Video document shows original material, and actual actors, but cockpit scenes are played by actors, based on cockpit voice recorder (another historic document)

Analysing history documents (continued) A combination of problems merged after the original hardware failure: functionality of the control system representation of the system towards the users training and information organisation problems human failure watch the video and try to identify problems in this task domain

/ONDERWIJS/PAO-NFatalErrorCDR.mov

Understanding implicit group knowledge Ethnographic methods No a priory conceptual framework bootstrapping - hanging around and identifying relevant concepts informal representations - diary and journalist type records activity theory, anthropology, sociology Goal: understanding complex settings multiple activities, multiple actors, multiple agendas in natural groups Focus on the situation, on work practice knowledge and meaning are developed in “communities of practice” knowledge is “ability to participate meaningfully” people co-construct knowledge and skills, partly implicit or “tacit” situated tasks, situated actions, situated understanding

Ethnographic methods (continued) Ethnography is fieldwork ethnographer is participant observer part of the situation, and co-constructor of events novice, if possible apprentice, or “legitimate peripheral participant” accepted to be “nosy” the two viewpoints of ethnographic analysis the view of the analyst: being surprised, recording novel phenomena the view of the aboriginal: understanding in situation and the relevance of resources and representations diary needed from day one

Ethnographic methods (continued) Ethnographic methods hanging around - identifying basic situational phenomena, getting accepted informal questions, triggered by activities and situation, based on growing understanding observation of routine and expert behaviour interaction analysis

Interaction analysis Focus on interactions (humans, artifacts, situation) identify "hot spots” –documents, stories, participant observation collect registrations - multiple instances content log –participants, situation, artifacts, technology, time –verbal and non verbal (body) language, territory, space, messages Video review sessions: comments of original participants explanations, thinking aloud ask to identify important issues propose hypotheses

Interaction analysis (continued) Interpretation: patterns in interaction, resources verifiable observations: records of different instances generalization: theory supplementary interviews, archive, observations Group sessions: interdisciplinary replays to generate hypotheses max 5 minutes for discussion identify analytic categories confront hypotheses with other instances of hot spot theory

Generalizing ethnographic knowledge Surprise index events and phenomena should be as predictable for the ethnographer as they are for the original actors Systematic coverage: space, territory, off location –make sure all relevant locations are considered (coffee lounge, corridors) temporal (shifts, seasonal changes, off hours) –related events during commuting, during lunch hour actors (supporting people, spouses, unofficial consultants)

Generalizing ethnographic knowledge (continued) Orientation - systematic viewpoints: choose one or more person oriented record –daily activities, interactions object oriented record –movements, change of ownership setting oriented record –at location, during shifts, movements in the space task / process oriented record –focus on process

Generalizing ethnographic knowledge (continued) event structure –preparation, start, re-arrangements, intro of new objects temporal patterns –day, season, technology driven rhythm –“acted” continuity (receptionist, pilot, underground train “driver”) interaction structure –verbal, instrumental, body language –interrupted conversation (continuation triggered by availability of info or resources or time) –just in time finalization triggered by interruption

Generalizing ethnographic knowledge (continued) spatial organization –meaning of positions (head of the table) –fixed places, free choice, stable or re-arrangements artifacts and documents –location and movements –construction and revision (documents) –ownership (allowed activities related to objects) –symbolic meaning (dress code, stethoscope) –non physical objects (PIN, password, anecdotes, jokes) ownership, trajectory, availability and inspectability

Example ethnographic record (video “Math Examination” VU 1994) Task analysis for the design of an automated examination system (preparing questionnaires, registering for exam, taking the exam, scoring results, administering marks) An examination: setting oriented record roles: professor, supervisors, students objects: forms, boxes, writing equipment, calculating equipment situation: time structure, entrance, scenes (hot spots) watch video and identify and analyse unexpected events

5.3. GroupWare task analysis (GTA) Conceptual framework for collecting and modeling task knowledge agents (with description of relevant attributes) actors (individual humans, institutes and groups, machines, systems) roles and their representation (incl. symbols) organization (structure of actors, roles, and allocation rules) work task / goal structure / actions (acts, communication acts) decomposition and temporal structure protocols and (expert) strategies - partly situated situation objects in object structure (type relations and semantic relations) environment history

GTA representations conceptual entities template structures tree flow media registrations video sound track picture

Task World Ontology (Martijn van Welie) Task Agent Role Event Object name(string) goal(string) start_condition(string) stop_condition(string) initial_state(string) final_state(string) duration(integer) frequency(string) type(enum) user_actions(string) system_operations(string) name(string) skills(string) attitude(string) miscellaneous(string) name(string) goal(string) name(string) attribute(Name,Value)* action(Name)* name(string) description(string) Contains Responsible Performed_by PlaysTriggers Subtask Uses Triggers Used_by Subrole Is Performed_by

EUTERPE GTA representations