1. Knowledge Enabled Information and Services Science Spatiotemporal and Thematic Semantic Analytics Matthew Perry PhD. Student Kno.e.sis Center, Wright State University

2. Knowledge Enabled Information and Services Science North Korea detonates nuclear device on October 9, 2006 near Kilchu, North Korea Thematic Dimension: What Temporal Dimension: When Spatial Dimension: Where Three Dimensions of Information

3. Knowledge Enabled Information and Services Science E1:Soldier E2:Soldier E3:Soldier E5:Battle E4:Address E6:Address E7:Battle occurred_at located_at lives_at assigned_to E8:Military_Unit assigned_to participates_in Georeferenced Coordinate Space (Spatial Regions) Dynamic EntitiesSpatial OccurrentsNamed Places Using named relationships to connect thematic entities with spatial locations in a variety of meaningful ways (different contexts) Residency Battle Participation E1:Soldier

4. Knowledge Enabled Information and Services Science Platoon_45 Soldier_2 Soldier_1 assigned_to:[3, 12] assigned_to:[6, 13] Soldier_4 Soldier_3 assigned_to:[19, 25] assigned_to:[14, 20] Temporal Properties of Paths Which soldiers were members of Platoon_45 during the interval [5, 15] ? Which soldiers were members of Platoon_45 at the same time ?

5. Knowledge Enabled Information and Services Science E5:Terroris t E6:Attack E4:Chemica l E10:Docto r E9:Base E8:Soldie r E7:Platoo n E1:Soldie r E3:Diseas e E2:Sympto m E14:Battl e E12:Platoo n E13:Soldie r E11:Locatio n assigned_to participated_in member_of carried_out spotted_at stationed_at member_of sign_of participated_in causes member_of used_in exhibits Near in Space After the Battle [4, 6] [8, 10] [0, 10] [3, 5] [0, 2] Spotted Close in Time Example: Bioterrorism

6. Knowledge Enabled Information and Services Science Background New types of applications exploiting named relationships between entities (semantic graphs) –Data Mining – Link Mining, Graph Mining –Semantic Web – Semantic Analytics Analysis of relationships in Large RDF graphs Detecting Conflict of Interest, Collaboration, Insider Threat Problem

7. Knowledge Enabled Information and Services Science Semantic Connectivity Two entities e 1 and e n are semantically connected if there exists a sequence e 1, P 1, e 2, P 2, e 3, … e n-1, P n-1, e n in an RDF graph where ei, 1  i  n, are entities and P j, 1  j < n, are properties &r1 &r5 &r6 worksFor “Matt” “Perry” fname lname Semantically Connected “Knoesis Center” name “Wright State University” name associatedWith

8. Knowledge Enabled Information and Services Science Semantic Similarity Two entities e 1 and f 1 are semantically similar if there exist two semantic paths e 1, P 1, e 2, P 2, e 3, … e n-1, P n-1, e n and f 1, Q 1, f 2, Q 2, f 3, …, f n-1, Q n-1, f n semantically connecting e 1 with e n and f 1 with f n, respectively, and that for every pair of properties P i and Q i, 1  i < n, either of the following conditions holds: P i = Q i or P i  Q i or Q i  P i ( means rdf:subPropertyOf ) We say that the two paths originating at e 1 and f 1, respectively, are semantically similar and thus so are the entities e 1 and f 1 Semantically Similar &r8 &r2 paidby “Fred” “Smith” &r7 &r1 fname lname purchased “Bill” “Jones” paidby &r9 fname lname &r3 Corporate AccountTicketPassenger

9. Knowledge Enabled Information and Services Science Spatial and Temporal Semantic Analytics 101 st Airborne 9 th SS Panzer 82 nd Airborne Axis Allies Position_2 Position_1 Position_3 Soldier_1 Soldier_2 Soldier_3 Explosives Soldier_5 Soldier_4 Camp Claiborne Held_position: [3, 7] Held_position: [4, 12] Held_position: [2, 10] Employed_unit Member_of Specializes_in Member_of Trained_at: [1, 5] Trained_at: [2, 6] Soldier_6Soldier_7 Member_of Trained_at: [8, 10] Spatial Relationship Thematic Relationship Temporal Relationship

10. Knowledge Enabled Information and Services Science Spatial Relationships Between Entities Examples: –Which Military Units have spatial extents which are within 20 miles of (48.45° N, 44.30° E) in the context of Battle participation? –Which infantry unit’s operational area overlaps the operational area of the 3 rd Armored Division? Quantitative RelationshipsQualitative Relationships

11. Knowledge Enabled Information and Services Science Temporal Relationships Between Entities Examples: –Which Speeches by President Roosevelt were given within one day of a major battle? –Who were members of the 101 st Airborne during November 1944? Quantitative RelationshipsQualitative Relationships

12. Knowledge Enabled Information and Services Science Current Work Define a Domain-independent Ontology which integrates Spatial and Thematic Knowledge –Allows exploiting the flexibility and extensibility of Semantic Web data models –Can deal with incompleteness of information on the web Incorporate temporal metadata into this model Identify and formalize basic spatial and temporal relationship-based query operators which complement current thematic operators of SemDis Matthew Perry, Farshad Hakimpour, Amit Sheth. "Analyzing Theme, Space and Time: An Ontology-based Approach", Fourteenth International Symposium on Advances in Geographic Information Systems (ACM- GIS '06), Arlington, VA, November 10 - 11, 2006

13. Knowledge Enabled Information and Services Science Upper-level Ontology modeling Theme and Space Occurrent Continuant Named_Place Spatial_Occurrent Dynamic_Entity Spatial_Region Occurrent: Events – happen and then don’t exist Continuant: Concrete and Abstract Entities – persist over time Named_Place: Those entities with static spatial behavior (e.g. building) Dynamic_Entity: Those entities with dynamic spatial behavior (e.g. person) Spatial_Occurrent: Events with concrete spatial locations (e.g. a speech) Spatial_Region: Records exact spatial location (geometry objects, coordinate system info) occurred_at located_at occurred_at: Links Spatial_Occurents to their geographic locations located_at: Links Named_Places to their geographic locations rdfs:subClassOf property Final Classification of Domain Classes depends upon the intended application

14. Knowledge Enabled Information and Services Science Occurrent Continuant Named_Place Spatial_Occurrent Dynamic_Entity Person City Politician Soldier Military_Unit Battle Vehicle Bombing Speech Military_Event assigned_to on_crew_of used_in gives participates_in trains_at Spatial_Region located_atoccurred_at Upper-level Ontology Domain Ontology rdfs:subClassOf used for integration rdfs:subClassOf relationship type

15. Knowledge Enabled Information and Services Science Thematic Context Specifies a type of connection between resources in the thematic dimension of our ontology ‘John Smith’ ‘B24#123’ ‘Bombing#456 ’ on_crew_ofused_in Person Military_Vehicl e Bombing on_crew_ofused_in Schema Person.on_crew_of.Military_Vehicle.used_in.BombingPath Template ‘John Smith’.on_crew_of.Military_Vehicle.used_in.Bombing

16. Knowledge Enabled Information and Services Science Thematic Query (ρ-theme) ρ-theme (G, t c )  {p t } Example: find all Bombing events connected to ‘John Smith’ through a vehicle participation context ρ-theme (G, ‘John Smith’.on_crew_of.Military_Vehicle. used_in.Bombing) ‘John Smith’.on_crew_of.‘B-24#123’.used_in.‘Bombing#456’ ‘John Smith’.on_crew_of.‘B-24#123’.used_in.‘Bombing#789’ Result G = temporal RDF Graph, tc = thematic context, p t = thematic context instance

17. Knowledge Enabled Information and Services Science E1:Soldier E2:Soldier E3:Soldier E5:Battle E4:Address E6:Address E7:Battle occurred_at located_at lives_at assigned_to E8:Military_Unit assigned_to participates_in Georeferenced Coordinate Space (Spatial Regions) Dynamic EntitiesSpatial OccurrentsNamed Places Thematic Contexts Linking Non-Spatial Entities to Spatial Entities Residency Battle Participation E1:Soldier

18. Knowledge Enabled Information and Services Science Incorporation of Temporal Information into RDF Use Temporal RDF Graphs defined by Gutiérrez, et al 1 Models Absolute Time Considers time as a discrete, linearly-ordered domain Associate time intervals with statements which represent the valid-time of the statement –Essentially a quad instead of a triple 1. Claudio Gutiérrez, Carlos A. Hurtado, Alejandro A. Vaisman: Temporal RDF. ESWC 2005: 93-107

19. Knowledge Enabled Information and Services Science Example Temporal Graph: Platoon Membership E1:Soldier E3:Platoon E5:Soldier E2:Platoon E4:Soldier assigned_to [1, 10] assigned_to [11, 20] assigned_to [5, 15]

20. Knowledge Enabled Information and Services Science Querying in the STT dimensions Provide a means to query about spatial, thematic, and temporal properties/relationships of all entities Path Query in the thematic dimension –Thematic Context Associate spatial region with a path Associate temporal interval with a path Query operators based on properties of and relationships between associated spatial regions and temporal intervals

21. Knowledge Enabled Information and Services Science Deriving Spatial and Temporal Properties of Entities ρ-spatial_extent (G, {p t })  {p t, sr} Retrieves the Spatial_Region connected (through occurred_at or located_at) to the terminating Spatial Entity of the context instances Example: Where were the battles in which the ‘101 st Airborne Division’ fought? ρ-spatial_extent (G, ρ-theme (G, ‘101 st Airborne Division’..participates_in.Battle)) ‘101 st Airborne Division’.participates_in. ‘Operation Market Garden’, ‘Geom#123’ Result G = temporal RDF Graph, p t = thematic context instance, sr = spatial region

22. Knowledge Enabled Information and Services Science Temporal Properties of Context Instances Platoon#456Soldier#789Soldier#123 assigned_to:[3, 12] assigned_to:[6, 20] Intersection [6, 12] Range [3, 20]

23. Knowledge Enabled Information and Services Science Deriving Spatial and Temporal Properties of Entities ρ-temporal_intersect ({p t })  {p t, [t 1, t 2 ]} Retrieves the interval during which the entire path is valid Example: Which Soldiers were members of the ‘1 st Armored Division’ at the same time? ρ-temporal_intersect (ρ-theme (G, Soldier.assigned_to.‘1 st Armored Division’.assigned_to.Soldier)) ‘Fred Smith’.assigned_to.’1 st Armored Division’.assigned_to. ‘Bill Jones’, [1941:04:15, 1943:02:30] Result p t = thematic context instance, [t 1, t 2 ] = temporal interval

24. Knowledge Enabled Information and Services Science Querying Relationships Between Entities (Thematic Context Instance t p, Temporal Interval [t i, t j ], Spatial Region sr) 3 Spatial Relationship Operators ρ-spatial_locate ρ-spatial_eval ρ-spatial_find 3 Temporal Relationship Operators ρ-temporal_restrict ρ-temporal_eval ρ-temporal_find Identify 6 major Spatiotemporal Relationship Queries which can be answered by combining previously defined operators

25. Knowledge Enabled Information and Services Science S 1  ρ-spatial_extent (G, ρ-theme (G, ‘101 st Airborne Division’. participates_in.Battle)) S 2  ρ-spatial_extent (G, ρ-theme (G, ‘1 st Armored Division’. participates_in.Battle)) ANS  ρ-temporal_intersect (ρ-spatial_eval (S 1, S 2, distance (S 1, S 2 ) 10 miles) Spatiotemporal Relationship Queries Example: When did the 101 st Airborne Division come within 10 miles of the 1 st Armored Division in the context of Battle participation

26. Knowledge Enabled Information and Services Science Group 1 Fred Account_1234 Jim 125 Broad Street Spatiotemporal Semantic Associations Define setting as a region of space in combination with an interval of time How is entity X related to Spatial setting S? ( ρ (entity, setting)) Attack Site How is Group 1 connected to the setting of the expected attack?

27. Knowledge Enabled Information and Services Science Spatiotemporal Semantic Associations Group 1 Fred Account_1234 Jim 125 Broad Street Group 2 How are entity X and entity Y related w.r.t Spatial setting S? ρ (entity, entity, setting) How are Group 1 and Group 2 connected with respect to the location of the crime?

28. Knowledge Enabled Information and Services Science Idea of Virtual Links between entities based on Spatiotemporal information Possible definition of rules to define a virtual link type –Collaboration: entity X and Y are in close ST proximity more often than a given threshold –Knows: entity X and Y are in close ST proximity regularly Spatiotemporal Semantic Associations

29. Knowledge Enabled Information and Services Science How do temporal relationships affect association semantics –2 works_for relationships (overlapping times, disjoint times, etc) Complex queries based on all 3 dimensions –Which location is the most likely storage facility for exfiltrated weapon material Thematic (correct capabilities, linked to correct people) Spatial (where was the material last seen) Temporal (how long can the material stay out of storage) Other Aspects