1. THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW The EyesWeb XMI multimodal platform Overview 5 March 2015

2. THE EYESWEB PLATFORM Conceived for supporting research on multimodal expressive interfaces and interactive systems. Widely employed for designing and developing real-time dance, music, and multimedia apps. Adopted by research centres, universities, industry, artists, and cultural institutions. Adopted in EU projects : o I 5FP IST: MEGA, CARE-HERE, MEDIATE o 6FP ICT: TAI-CHI, Enactive, Humaine o I 7FP ICT: SAME, SIEMPRE, I-SEARCH, MIROR, HILARE, o ASC-Inclusion o I Culture 2007 Project CoMeDiA o I Culture Project MetaBody (metabody.eu) Actually adopted in the Horizon2020: ICT DANCE Distributed for free: www.infomus.org THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

3. OVERVIEW Includes libraries for audio and video processing. Includes libraries for movement analysis. Includes support for commercial applications. Can be freely extended by third parties. Supports consolidated industrial standards e.g., Steinberg VST and ASIO (support to audio multi-channel h/w boards), OSC, MIDI, TCP/IP, serial, firewire, USB webcam,wide range of sensors and frame grabbers, etc. THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

4. OVERVIEW (II) EyesWeb consists of a number of integrated hardware and software modules which can be easily interconnected and extended. The EyesWeb software includes a development environment and a set of libraries of reusable software components (blocks), which can be assembled by the user in a visual language to build applications (patches). EyesWeb also supports multiple sensors: e.g. video-cameras, microphones, and biometric sensors, such as on-body wireless sensor systems (e.g., accelerometers, physiological sensors). THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

5. OVERVIEW (III) Suited for: Multimodal systems: interaction through different communication channels, automatic extraction and convention of meaning. Multimedia systems: multimedia data management (e.g. audio, video) both live (video-cameras, microphones) or archived (avi, mpeg files). Real-time systems: o Time constrained due to interaction with sensors; o Time constrained on feedback to the user (otherwise the system is not perceived as interactive). THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

6. THE FRAMEWORK ARCHITECTURE THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

7. THE EYW CONSOLE THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

8. THE EYW CONSOLE (II) Features: Runs patches from the command line to reduce the GUI overhead. On Windows, it runs both as a standard application or as a Windows service. Can act as server for EyesWeb Mobile. It is used to run patches in Linux. THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

9. THE EYW CONSOLE (II) THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

10. THE EYW GDE THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

11. THE EYW GDE (II) The EyesWeb Graphic Development Environment (GDE) is used to design, develop, and run EyesWeb applications (patches). A patch is a structured network of EyesWeb modules (blocks) that channels and manipulates a digital input dataflow, resulting in a desired output dataflow. It is based on a visual language paradigm. THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

12. THE EYW GDE (III) The EyesWeb Graphic Development Environment (GDE) is used to design, develop, and run EyesWeb applications (patches). A patch is a structured network of EyesWeb modules (blocks) that channels and manipulates a digital input dataflow, resulting in a desired output dataflow. It is based on a visual language paradigm. THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

13. THE EYW QUERY THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

14. THE EYW QUERY (II) Tool for generating documentation from an EyesWeb installation. Can generate the full documentation set or just the documentation of the specified blocks. Generates latex (pdf), text, MySql documentation. THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

15. THE EYW QUERY (II) THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

16. THE EYW REGISTER MODULE THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

17. THE EYW REGISTER MODULE (II) Tool to registering DLLs into the EyessWeb environment. Every new block is a DLL that need to be registered THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

18. DISTRIBUTED TOOLS THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

19. THE EYESWEB MOBILE DESIGNER AND RUNTIME These tools allow to create independent EyesWeb interfaces Can control one or more patches from many different devices THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW

20. INFO AND CONTACTS Videos: www.youtube.com/InfoMusLab Papers: ftp.infomus.org/pub/Staff/AntonioCamurri Web Portal: www.casapaganini.orgwww.casapaganini.org Research team: Antonio Camurri, Gualtiero Volpe, Corrado Canepa, Paolo Coletta, Nicola Ferrari, Simone Ghisio, Stefano Piana, Paolo Albrono, Ksenia Kolykhalova, Maurizio Mancini, Alberto Massari, Radoslaw Niewiadomski. THE EYESWEB PLATFORM - OVERVIEWTHE EYESWEB PLATFORM - OVERVIEW