1. MyActivity: A Cloud-Hosted Ontology-Based Framework for Human Activity Querying Amin BakhshandehAbkear Email: abakhshandehabk@students.latrobe.edu.au Supervisor: Associate Professor. Seng W. Loke LaTrobe University

2. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Outline 1.Introduction 2.Background 3.Generic Architecture and Design 4.Prototype Implementation 5.Examples and Evaluation 6.Conclusion and Future Works

3. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Introduction Pervasive Computing –Context-Aware Activity Recognition (AR) –ADL (Activities of Daily Living) Aims –Ontology + Stream Reasoner  Complex Activity Recognition –Developing an Android API Sensors Raw data retrieval Preprocessing/reasoning Storage/management application Sensing subsystem Thinking subsystem Acting subsystem Abstract layered architecture for context-aware systems [1]

4. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Background Activity Recognition (AR) –General Architecture –Approaches Knowledge-Based  Logical modelling  Rule based (Logic programming, spatial temporal logic, Fuzzy logic, Evidence theory and Ontology ) Data-Driven (Machine learning)  Statistical and probabilistic (HMM, Neural Network, Decision Tree, Bayesian Network ) Hybrid Model Abstract process for situation detection [2]

5. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Background Ontology –Presenting domain-knowledge –Sensor data into well-defined vocab – Reusable, Sharable, Undrestandable (?user rdf:type socam:Person), (?user, socam:locatedIn, socam:Bedroom), (?user, socam:hasPosture, ‘LIEDOWN’), (socam:Bedroom, socam:lightLevel, ‘LOW’), (socam:Bedroom, socam:doorStatus, ‘CLOSED’) -> (?user socam:status ‘SLEEPING’) (?user rdf:type socam:Person), (?user, socam:locatedIn, socam:Bedroom), (?user, socam:hasPosture, ‘LIEDOWN’), (socam:Bedroom, socam:lightLevel, ‘LOW’), (socam:Bedroom, socam:doorStatus, ‘CLOSED’) -> (?user socam:status ‘SLEEPING’) Situational context ontology [3]

6. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Background Semantic Data Concepts –RDF (Resource Description Framework) Triple : –RDFS (RDF Schema) –OWL (Web Ontology Language) –SPARQL (Simple Protocol and RDF Query Language) http://amin.ontology.com#Richard, http://amin.ontology.com#isPerforming, http://amin.ontology.com#walking <rdfs:label rdf:datatype="http://www.w3.org/2001/XMLSchema#string" >OnBus

7. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Background C-SPARQL (Stream Reasoner) Quadruple (RDF STREAM) : (, TimeStamp) window input streams streams of answer Registered Continuous Query

8. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Generic Architecture and Design Sensor/GPS Environment Message Builder GUI / AR notifier Event Handler Internet Feature Extractor Classifier Buffer Mobile Device (Client) Cloud Web Service (Server) Messaging Service Stream Reasoner Engine Atomic Activity Receiver RDF Stream Builder Ontology Activity Component detector Result Publisher Stream Gate Register Query Raw Data Atomic Activity Realized quadruples

9. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Ontology Design Generic Architecture and Design

10. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Ontology Design Generic Architecture and Design Proposed Ontology for Complex Activity Recognition

11. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Ontology Design for Complex Activity CommutingOnBus Generic Architecture and Design

12. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Generic Architecture and Design Exercise Program

13. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Implementation Developing Data Collector App Developing Feature Extractor Training Data Set and Building Classifiers Building Ontology Developing Client App (Android-based) Developing Web Service (C-SPARQL) Cloud Services –Amazon Elastic BeansTalk –Amazon SQS (Simple Queue Service)

14. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Query types –Simple Queries –Aggregate Queries –Complex Queries (Ontology + Static Knowledge) REGISTER QUERY SeeRawStream AS PREFIX amin_ontology_c: PREFIX rdf: SELECT ?atype (COUNT(?atype) AS ?activities) FROM STREAM [RANGE 5m STEP 5s] WHERE { ?s amin_ontology_c:isPerformingActivity ?a1. ?a1 rdf:type ?atype } GROUP BY ?atype ORDER BY ?activities Examples and Evaluation [http://amin.ontology.com#Running"5"^^http://www.w3.org/2001/XMLSchema#integer] [http://amin.ontology.com#Walking"10"^^http://www.w3.org/2001/XMLSchema#integer] [http://amin.ontology.com#Inactive"65"^^http://www.w3.org/2001/XMLSchema#integer]

15. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Response Time Calculation Examples and Evaluation (1) (2) (3) (1), (2), (3)  T(avg) = 12 sec

16. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Examples and Evaluation Server Execution time RANGESTEP Average Obtained Time (sec) = A Total Time (sec) = T 1 min5 sec0.636025.36 1 min2 sec0.612363.42 1 min10 sec0.681812.95 5 min5 sec1.307449.68 5 min10 sec1.298123.36 Table - Execution Time Obtained from Running a Complex Query Avg Obtained Time = number of times that query returns result over stream (19) Sum of execution time (12.95)

17. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University Conclusion Summary and Contributions –Using OWL Ontology; –Exploiting C-SPARQL engine as stream reasoner –Using hybrid model (Ontology, Machine-learning) –Enabling collaborative activity recognition Future Works –Adding Spatio and Temporal properties –Collaboration with Social Networks –Moving toward ‘Socially Aware Computing’

18. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University References : [1] S. W. Loke, “Incremental awareness and compositionality: A design philosophy for context-aware pervasive systems,,” Pervasive and Mobile Computing, vol. 6, no. 2, pp. 239-253, April 2010. [2] P. Wustmann, I. Braun, W. Dargie, M. Berger, F. Sandnes, Y. Zhang, C. Rong, L. Yang and J. Ma, “A Comprehensive Approach for Situation-Awareness Based on Sensing and Reasoning about Context,” in Ubiquitous Intelligence and Computing, Springer Berlin / Heidelberg, 2008, pp. 143-157. [3] L. Chen, I. Khalil, L. Chen, C. D. Nugent, J. Biswas and J. Hoey, “Activity Recognition: Approaches, Practices and Trends,” in Activity Recognition in Pervasive Intelligent Environments, Atlantis Press, 2011, pp. 1-31.

19. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University

20. physical: a given number of triples logical: a variable number of triples which occur during a given time interval (e.g., 1 hour) –Sliding: they are progressively advanced of a given STEP (e.g., 5 minutes) –Tumbling: they are advanced of exactly their time interval

21. A Cloud-Hosted Ontology-Based Framework for Human Activity Querying – 2012 – LaTrobe University