1. Human Robot Interaction in Guardians Amir M. NaghshSheffield Hallam University, UK Jeremi Gancet Space Applications Service N.V., Belgium Andry TanotoHeinz Nixdorf Institute, Paderborn, Germany Jacques Penders Sheffield Hallam University, UK Chris R. Roast Sheffield Hallam University, UK Michel Ilzkovitz Space Applications Service N.V., Belgium RISE 2008 Worksop, Benicassim www.guardians-project.eu

2. Outline 1.Introduction and context: Guardians environment 2.On-the-field HRI 3.Remote HRI 4.Communication issues 5.HRI metrics 6.Conclusion and future work www.guardians-project.eu

3. 1. Guardians’ Environment / Fire- fighters operations Environmental Conditions – No Visibility – Reduced sense of hearing – Restricted sense of touch

5. 1. Guardians’ Environment / Fire- fighters operations Fire-Fighters Operations – Fire-fighters safety – Teams of 2 (3,4) – Progress along a wall when it is possible – Frequently debrief the Entry Control Officer (ECO) – Span of control (Overall structure) ADD DIAGRAM ???

6. 2. HSI Requirements The swarm notifies the humans of possible hazards (e.g. obstacles, high temperature, chemicals) The swarm stays within a relatively close range but also maintains its distance to the fire fighters to allow them freedom of action The swarm indicates unambiguously the direction to the scene of incident or backwards to the exit point. The interfaces indicating hazards should be most noticeable for the fire fighters.

7. 2. Swarm  Human Interactions Visual – Visual presentation using LED / LCD screens – Providing Navigational information / Hazards Audio – Dialogue based interactions / Speech recognitions – Not recommended

8. 2. Swarm  Human Interactions Tactile – Using sense of touch for presenting information – Hazards – Obstacles / Navigation information Vibrators

9. 2. Human  Swarm Interactions Passive – Autonomous – form a bubble surrounding fire fighters – maintain a min/max distance Tangible – Big buttons – high level tasks Movement-based – Gesture recognition Audio / Visual – Not recommended VPL Research, 1987 MIT, 2007

10. 3. Remote HRI: the base station Base station’s purpose – Monitoring and control of the robots and humans on the field – Offline and online support to mission planning, coordination, scheduling and execution – Data recording and replay with support to debriefing Users roles – Different types of users, with different roles 1.Guardians’ appliance coordinator 2.Operator(s) 3.Sensor data specialist(s) 4.Stakeholders in the commanding chain 5.Tier information channels (media...) www.guardians-project.eu

11. 3. Remote HRI: the base station Base station components www.guardians-project.eu MTE and MPSEM Base Station Core MDRD HMI Clients Base Station SDP Users Interfaces to VF robots, HCM, and their sensors

12. 3. Remote HRI: the base station Base station components www.guardians-project.eu MTE and MPSEM Base Station Core MDRD HMI Clients Base Station SDP => HMI Servers => Base station administration / supervision

13. 3. Remote HRI: the base station Base station components www.guardians-project.eu MTE and MPSEM Base Station Core MDRD HMI Clients Base Station SDP => Mission edition support tools => Planning and scheduling => Execution monitoring

14. 3. Remote HRI: the base station Base station components www.guardians-project.eu MTE and MPSEM Base Station Core MDRD HMI Clients Base Station SDP => Mission data recording / replay => Mission data dispatching

15. 3. Remote HRI: the base station Base station components www.guardians-project.eu MTE and MPSEM Base Station Core MDRD HMI Clients Base Station SDP Sensor Data Processing => Ground based (cpu-consuming) data processing => Data fusion and map building => Robot localization

16. 3. Remote HRI: interfaces www.guardians-project.eu Nominal baselineConvenience / backupRestricted to well- scoped purpose Advanced visualization, extension of “ecological display” approach Joystick Touchscreen HHI: sound-based Mouse Keyboard Immersive VR HRI: sound-based Foreseen interfaces & approach to remote HRI: ADD ILLUSTRATIONS HERE ???

17. 4. Communication Issue Short range Communication Middle range Communication Issues: Interferences due to warehouse infrastructures No communication infrastructure is available at the incident Communication protocol Communication bandwidth Issues: Interferences due to warehouse infrastructures No communication infrastructure is available at the incident Communication protocol Communication bandwidth

18. 4. Wireless Communication Three wireless communication standards: WLAN (802.11) – High datarate, high power consumption, point to point connection – For high-rate data transfer: video, audio Bluetooth – Middle datarate, low power consumption, point to multi-point connection – For continuous, middle-rate data transfer: audio, sensors ZigBee – Low datarate, low power consumption, fast network formation time, high number of nodes – For time-critical low-rate data transfer: critical data (commands, sensor) Which, When, and Where?

19. 5. Validation & Evaluation What to measure? Situational awareness – To understand the extreme condition of the environment and robots Cognitive/Work load – To ensure load below humans’ capability Team performance – The outcome of teamwork between robots and humans How to measure? Interaction debugger [Kooijmans & Al, 2006] – to collect, match, visualize, and analyze SAGAT – to measure situational awareness Wizard of Oz www.guardians-project.eu

20. 6. Conclusion and future work HSI is a novel research area We presented: – issues on human-robot swarm interaction for search and rescue mission – System architecture to support human-robot swarm interaction We proposed – human-robot swarm interface – Metrics for validation and evaluation Future work to provide conceptual design of human- robot swarm interaction Interface prototyping and testing is ongoing with participation of end-users www.guardians-project.eu

21. Thank you for your attention!