1. INTERACT : M OTION S ENSOR D RIVEN G ESTURE R ECOGNITION C LOUD S ERVICE School of Electronic & Computer Engineering Technical University of Crete, Greece Dr. Stelios Sotiriadis K. Stravoskoufos, A. Preventis, E. Petrakis

2. Data! 640K ought to be enough for anybody (1981). …Google processes 20 PB = 2.19902326 × 10 13 kilobytes a day

3. Cloud Computing

4. The Internet of Things (IoT)

6. Cloud in Europe: FIWARE Open cloud infrastructure for cost-effective creation and delivery of FI applications and services Develop Cloud Services called “Generic Enablers”. EU FP7 project. Offers open specification services featuring powerful but user-friendly APIs based on RESTFul standards.

7. Generic Enablers (GEs) Speeds up the creation of FI applications. Cloud-based modular components which offer reusable and commonly shared functions. Serve multiple areas of use across various Sectors e.g. health! The fundamental building blocks of FIWARE.

8. FI-STAR FP7 FI-STAR Experimentation Sites and Use Case Scenarios. Servicing around 6 Million people throughout Europe. Work in parallel to apply and validate the Core FI- STAR Platform and its subset of GEs. www.fi-star.eu

9. XIFI FP7 Crete Node

10. XIFI FIWARE Node Crete CPU : 208 Cores RAM: 512 TB HDD: 11.7 TB IaaS services FIWARE Instances Blueprint instances SaaS APIs Feel free to contact me for requesting information!

11. Interact! Motion Sensors Cloud Driven Gesture Recognition Cloud open source software based on FIWARE Generic Enablers. It operates FIWARE cloud platform and offer their functionality through RESTful APIs.

12. Interact Architecture

13. IoT Connectivity & Protocol Adapter Connecting sensor with the FI application components. Uses the Protocol Adapter module to adapt to the specific connectivity protocol that the sensor is using (e.g., Bluetooth).

14. Sensor Data Collector Collects the sensor data Converts data into the desirable format (e.g., JSON, XML etc.) and schema so it can be processed in the back-end without causing interoperability problems.

15. Connectivity Service Establishes a connection between the front-end and the back-end Transfers the sensor data received from the «Sensor Data Collector» to the Back-End

16. Application Logic Application specific module. Encapsulates the business logic of the FI- application. Handles and processes sensor data by using the Complex Event processing, Cloud storage and Publish subscribe modules

17. Complex Event Processing Used for decision-making thorough the analysis of complex conditional events. Processes custom event patterns Based on user defined conditions, decides the flow of the data.

18. Cloud Storage Responsible for storing or retrieving sensor data using a database. Functionality should be offered as a RESTful service. Developers should be able to create or retrieve gestures from the database

19. Publish Subscribe Responsible for publishing the results of the sensor data processing. Publishes context to context subscribers. Example:

20. Identity Management Used for user authentication and access authorization. Users and Developers authorized to access applications and services

21. Motion Sensors Sensors that can track the human body or parts of it (skeletal tracking). Asus Xtion Pro MS Kinect Leap Motion

22. Interact FI application, based on the FIWARE core platform Example functionality: Gesture Recognition. GESTURE MOTION SENSORINTERACTRESULT “Victory"

23. Interact Functionalities Recognize Gestures: Static, Motion, Drawing Store Gestures: Developers can use interact to store gestures & create their own gesture pools.

24. Interact Functionalities Subscribe to gesture pools: Developers can subscribe to other developers gesture pools in order to use them in their projects. Create gesture pools: Public gesture pool - Can be accessed by everyone - Only super administrators can add new gestures Custom gesture pools - Created by developers

25. Interact Functionalities Train the System: Interact features a machine learning mechanism that allows users to train the system to recognize gestures. Implements Naive Bayes Classification Accuracy ≈ 75%

26. Interact Architecture

27. Use Case 1 Gesture to voice: Using Google text to speech API to transform the recognized gestures into voice in order to enable communication with people with speaking disabilities (e.g., deaf-mute).

28. Use Case 2

29. Use Case 2 Demo

30. Use Case 2 Architecture

31. Conclusions & Future Work Highlights new means of communication for people with special needs. Interact is online! Ask for more details. Try different machine learning algorithms to increase gesture recognition accuracy.

32. THANK YOU! Questions ?