___________________________________________________ Intelligent Planning and Collaborative Systems for Emergency Response UNIQUE AIAI RESOURCES More than 20 years of excellence in applied Artificial Intelligence World-leading AI planning research and technical team World-leading knowledge modelling and representation resources and staff O-Plan: Multi-Perspective Planning Architecture and Planning Web Service I-X: Issue Handling Planning and Collaboration Architecture : Knowledge Elicitation, Encoding, Modelling, Representation, and Management I-X commercialisation through Scottish Enterprise Proof-of-Concept Award: IM-PACs This briefing is available in

2 Edinburgh AI Planners in Productive Use

AUTOMATED REASONING AIAI TECHNOLOGIES KNOWLEDGE MODELLING I-X: Issue Handling and Task Support Architecture DECISION MAKING RESPONSE TEAM Constraints Issues Nodes Space of Legitimate Solutions Issues or Implied Constraints Node Constraints Detailed Constraints I N C A=Annotations Do (IH) Choose (IH) IH=Issue Handler (Agent Functional Capability) Propagate Constraints Planning System Intelligent Messaging, Planning and Collaboration Systems for Emergency Response Knowledge about places, people, processes, infrastructure, connectivity, response capabilities, and meta-knowledge : Knowledge Elicitation, Encoding, Modelling, Representation, and Management EVENT DRAFT RESPONSE PLANS: MULTIPLE COURSES OF ACTION Effects-Oriented Planning O-Plan/I-Plan: Multi-Perspective Planning GOALS TASKS COMMUNICATION COORDINATED RESPONSE KNOWLEDGE BASE Shared Task and Activity Model

4 A More Collaborative & Dynamic Planning and Execution Framework  Human relatable and presentable objectives, issues, sense- making, advice, multiple options, argumentation, discussions and outline plans for higher levels  Detailed planners, search engines, constraint solvers, analyzers and simulators act as services in this framework in an understandable way to provide feasibility checks, detailed constraints and guidance  Sharing of processes and information about process products between humans and systems  Current status, context and environment sensitivity  Links between informal/unstructured sense-making and discussion and more structured planning, methods for optimisation and decision support

5 I-X Multi-Agency Emergency Response Planning, Execution, and Task-Oriented Communications Collaboration and Communication Command Centre Central Authorities Isolated Personnel Emergency Responders

6 Framework  Common conceptual basis for sharing information on processes and process products  Shared, intelligible to humans and machines, easily communicated, formal or informal and extendible  Set of restrictions on things of interest: IIssuese.g. what to do? How to do it? NNodese.g. include activities or product parts CConstraintse.g. state, time, spatial, resource, … AAnnotationse.g. rationale, provenance, reports, …  Shared collaborative processes to manipulate these: Issue-based sense-making (e.g. gIBIS, 7 issue types) Activity Planning and Execution (e.g. mixed-initiative planning) Constraint Satisfaction (e.g. AI and OR methods, simulation) Note making, rationale capture, logging, reporting, etc.  Maintain state of current status, models and knowledge  I-X Process Panels (I-P 2 ) use representation and reasoning together with state to present current, context sensitive, options for action Mixed-initiative collaboration model of mutually constraining things

7  The I-X approach involves the use of shared models for task- directed communication between human and computer agents  I-X system or agent has two cycles: Handle Issues Manage Domain Constraints  I-X system or agent carries out a (perhaps dynamically determined) process which leads to the production of (one or more alternative options for) a “product”  I-X system or agent views the synthesised artifact as being represented by a set of constraints on the space of all possible artifacts in the application domain I-X Approach

8 Constraints Issues Nodes Product Model Space of Legitimate Product Models Issues or Implied Constraints Node Constraints Detailed Constraints I N C A Annotations

9 Constraints Issues Nodes Product Model Space of Legitimate Product Models Issues or Implied Constraints Node Constraints Detailed Constraints I N C A Annotations Do (IH) Choose (IH) IH=Issue Handler (Agent Functional Capability) Propagate Constraints I-X and

10 I-P 2 aim is a Planning, Workflow and Task Messaging “Catch All”  Can take ANY requirement to: Handle an issue Perform an activity Respect a constraint Note an annotation  Deals with these via: Manual activity Internal capabilities External capabilities Reroute or delegate to other panels or agents Plan and execute a composite of these capabilities (I-Plan)  Receives reports and interprets them to: Understand current status of issues, activities and constraints Understand current world state, especially status of process products Help user control the situation  Copes with partial knowledge of processes and organisations

11 Anatomy of an I-X Process Panel

12 Process Panel I-X Process Panel and Related Tools Domain Editor Messenger I-Plan Map Tool

13 I-Space and I-World

14

15 Safety and Companion Robots

16 e-Response Vision  The creation and use of task-centric virtual organisations involving people, government and non-governmental organisations, automated systems, grid and web services working alongside intelligent robotic, vehicle, building and environmental systems to respond to very dynamic events on scales from local to global.  Multi-level emergency response and aid systems  Personal, vehicle, home, organisation, district, regional, national, international  Backbone for progressively more comprehensive aid and emergency response  Also used for aid-orientated commercial services  Robust, secure, resilient, distributed system of systems  Advanced knowledge and collaboration technologies  Low cost, pervasive sensors, computing and comms.  Changes in building codes, regulations and practices

17 e-Response Relevant Technologies  Sensors and Information Gathering sensor facilities, large-scale sensor grids human and photographic intelligence gathering information and knowledge validation and error reduction semantic web and meta-knowledge simulation and prediction data interpretation identification of "need"  Emergency Response Capabilities and Availability robust multi-modal communications matching needs, brokering and "trading" systems agent technology for enactment, monitoring and control  Hierarchical, distributed, large scale systems local versus centralized decision making and control mobile and survivable systems human and automated adjustable autonomy mixed-initiative decision making mixed-initiative, multi-agent planning and control trust, security  Common Operating Methods shared information and knowledge bases Shared standards and interlingua shared human scale self help web sites and collaboration aids shared standard operating procedures at levels from local to international standards for signs, warnings, etc.  Public Education publicity materials self help aids public training

18 FireGrid Technologies Maps, Models, Scenarios Super-real-time Simulation Knowledge Systems, Planning & Control Emergency Responders Computational Grid Tens of Thousands of Sensors & Monitors

19 FireGrid Overview  Mission statement: -…to establish a cross-disciplinary collaborative community to pursue fundamental research for developing real time emergency response systems using the Grid… -Initial domain is fire emergencies.  Challenges: -Sensing: instantaneous and continuous relay of data from emergency location to response system via the Grid. -Modelling: model the evolution of fire and impact on building, and relate this to intervention alternatives and evacuation strategies. -Forecast: all simulations, analyses and communications done in ‘super real-time’. -Response: effective co-ordination of response with intelligent decision-support system. -Feedback: continuously update simulations, predictions and response using latest data from sensors and responders.  Status: -DTI/University of Edinburgh/Industry-funded project, total value: £2.23M, start date: 1 st March Modelling Emergencies in Real-Time from Sensor Input (MERSI) research project at initial (EPSRC) proposal stage.

20 The FireGrid Cluster  Universities and Colleges: -University of Edinburgh; Imperial College London; Queen Mary, University of London; The Fire Service College, UK; Institute of High Performance Computing, Singapore; TU Delft, The Netherlands; IHMC Florida  National Research Laboratories: -National e-Science Centre, UK; Health and Safety Laboratory, UK; NIST, USA; Major Accident Prevention Division, IRSN, France; TNO Building and Construction Research, The Netherlands.  Computational Software and Sensing Technology Companies: -Vision Systems (Europe) Ltd.; ABAQUS UK Ltd.; ANSYS Europe Ltd.; Integrated Environment Solutions Ltd.  Engineering and Technology Consultancy Companies: -Arup Fire; BRE Building Research Establishment Ltd.  Emergency Planning and Response: -Fire Research Division, Office of the Deputy Prime Minister, UK; London Fire and Emergency Planning Authority; Lothian and Borders Fire Brigade, Edinburgh; Greater Manchester County Fire and Rescue Service.

21 Adapted from H. Kitano and S. Tadokoro, RoboCup Rescue A Grand Challenge for Multiagent and Intelligent Systems, AI Magazine, Spring, Cycle 20 Cycle 200 Blocked RoadsRoadsBuildings Ambulance TeamFire BrigadePolice Force Ambulance Centre Fire Station Police Office Search and Rescue Command Centre RoboCup Rescue Simulator Simulates the Kobe earthquake Sends sensorial information to agents, receiving back action commands I-X Agents Divided in three hierarchical decision-making levels Support ideas such as activity oriented planning, coordination and knowledge sharing Interaction I-X to Kobe Simulator Information from RCRS to I-X is converted to the format

Galileo

24 More Information i-rescue.org i-x.info i-c2.com

25 Prof. Austin Tate Technical Director, Artificial Intelligence Applications Institute Professor of Knowledge-Based Systems, University of Edinburgh Fellow of the Royal Society of Edinburgh (Scotland's National Academy), Fellow of the American Association for AI, Fellow of the British Computer Society, Fellow of the International Workflow Management Coalition, and a member of the editorial board of a number AI journals. His internationally sponsored research work involves advanced knowledge and planning technologies, especially for use in emergency response and search and rescue..

26 Spare Slides  Spare Slides

27 High Level Planning and Activity Management Sensors, User Inputs, , External Influences Behaviours: Preprogramed, Situation-Response, Reactive Sub-plan Library HTN Planning & Diary

28 HTN Planning Activity Composition A1 A2 A3 A5 A4 “Initial” Plan Refine Introduce activities to achieve preconditions Resolve interactions between conditions and effects Handle constraints (e.g. world state, resource, spatial, etc.) “Final” Plan A2.2 A2.1 A1 A3 A5 A4 Plan Library A2 Refinement S2 S1

29 HTN Planning Initial Plan Stated as “Goals” Refine Plan Library Ax Refinement S2 S1 P Initial Plan can be any combination of Activities and Constraints “Refined” Plan A1.2 A1.1 Q P “Initial” Plan P Q

30 Some Planning Features  Expansion of a high level abstract plan into greater detail where necessary.  High level ‘chunks’ of procedural knowledge (Standard Operating Procedures, Best Practice Processes, Tactics Techniques and Procedures, etc.) at a human scale - typically 5-8 actions - can be manipulated within the system.  Ability to establish that a feasible plan exists, perhaps for a range of assumptions about the situation, while retaining a high level overview.  Analysis of potential interactions as plans are expanded or developed.  Identification of problems, flaws and issues with the plan.  Deliberative establishment of a space of alternative options, perhaps based on different assumptions about the situation involved, of especial use ahead of time, in training and rehearsal, and to those unfamiliar with the situation or utilising novel equipment.

31 More Planning Features  Monitoring of the execution of events as they are expected to happen within the plan, watching for deviations that indicate a necessity to re-plan (often ahead of this becoming a serious problem).  Represent the dynamic state of the world at points in the plan and use this for ‘mental simulation’ of the execution of the plan.  Pruning of choices according to given requirements or constraints.  Situation dependent option filtering (sometime reducing the choices normally open to one ‘obvious’ one.  Satisficing search to find the first suitable plan that meets the essential criteria.  Heuristic evaluation and prioritisation of multiple possible choices within the constrained search space.  Uniform use of a common plan representation with embedded rationale to improve plan quality, shared understanding, etc.

32 Human Approach  Previous slides describe aspects of problem solving behaviour observed in expert humans working in unusual or crisis situations.  Gary Klein, “Sources of Power”, MIT Press,  But they also describe the hierarchical and mixed initiative approach to planning in AI developed over the last 25 years.

Compendium

35 Compendium

36 Ontology Issues Outstanding questions, problems or requirements (gIBIS) Nodes E.g. activities in a process or parts in a physical product Constraints Critical Constraints (shared across multiple components) Auxiliary Constraints (localised to a single component) Annotations E.g. decision rationale and other notes