1. © Ramesh Jain Ramesh Jain CTO, PRAJA inc. and Professor Emeritus, UCSD rjain@praja.com Emergent Semantics and Experiential Computing

2. © Ramesh Jain Semantics Meaning Negotiation process: requires agreement among all involved participants Every User has their ‘personal ontology’ Personal ontology can not be matched with the system ontology in one step (one query) --- requires an emergent process.

3. © Ramesh Jain Powerful Data Models: Relational Information Sources VideoAudio Sensors Data and Statistics Text Index XML Feature Indexed Early Developments

4. © Ramesh Jain The Semantic Gulf Data is organized based on data characteristics. Efficiency and Scalability are primary User intentions and context are not considered. Query environment does not maintain the state of the user.

5. © Ramesh Jain Access: Impedance Mismatch Computers are millions of times faster than humans in arithmetic and logic. People are millions of times better than computers in perceptual and conceptual tasks. Any one year old can recognize objects! Current computing environments were designed for people to serve computers. Consider computers and humans part of a symbiotic system.

6. © Ramesh Jain Semantic Indexing VideoAudioSensors Data and Statistics Text Index Index and Link based on Event-graph

7. © Ramesh Jain Information Assimilation User EventBase Model- Based Updating And Linking Navigation and Visualization Environment. Semantic procedural updating Semantic links used to display unstructured data

8. © Ramesh Jain Features of Experiential Environment Natural Action Responses – No unusual metaphors Query and Presentation spaces must be the same (What-You-See-Is-What-You-Get) Continuity of User State and Context – Minimize Latency; Feedback Multimedia Immersion and Exploration Video games should be the model.

9. © Ramesh Jain Example Applications

10. © Ramesh Jain More than Five Century old Legacy of Gutenberg Continues… Despite all advances in technology… DocumentWeb: Information Age

11. © Ramesh Jain Strategic Inflection Points Documents on Web (Information) Events on Web (Experience) 1995 200020052010 Keyword Search Semantic Search Contextual Search Immersive Experience Updates and alerts Ubiquitous Devices

12. © Ramesh Jain EventWeb: Experience Age Family Sports Fun Knowledge Personal Finance Office

13. © Ramesh Jain Top 5 Misconceptions All users should have Ph.D. in Ontology. User queries are Context-Free. Users must be allowed only one query. Image and Video semantics is in features of FULL images or video. All information is alpha-numeric.

14. © Ramesh Jain Thanks.

15. © Ramesh Jain Data Experience Information

16. © Ramesh Jain Information Integration: 1 Query Parsing And Report Generation User

17. © Ramesh Jain

18. Time Machine - Replay Example Apps

19. © Ramesh Jain Information System Evolution: Databases Query Translation And Response Users Queries Declarative and Stateless

20. © Ramesh Jain Information System Evolution : Personalization Query Translation And Response Users P- Queries Personalization Filters Queries Declarative and Stateless

21. © Ramesh Jain Information System Evolution : Contextual Systems Query Translation And Response Users P- Queries Personalization Filters QueriesContext User States Declarative and Stateless

22. © Ramesh Jain Contextual Navigation Context Data Presentation System Query Transformation What-You-See-Is-What-You-Get (WYSIWYG) Search.

23. © Ramesh Jain Kalman Filtering Processing Model Data

24. © Ramesh Jain Basic Concept Model: Mathematical State and other relations Current State = F (Previous State, New Observations) Models states and transitions based on each source of data

25. © Ramesh Jain Unified Indexing Should allow indexing into all data and information sources Should be independent of a medium Should not use idiosyncrasy of a medium Video: time, frame number; Text: page, line;...

26. © Ramesh Jain Symbolic Kalman Filtering The Model is Hybrid – combination of symbolic and mathematical. Each Data Source is an independent observation source. Model is updated based on its current state and new observations. Essential for live and sensory data assimilation.

27. © Ramesh Jain Entities and Events Events are dynamic. Event Name Duration Location Attributes Data-streams Processes Adjacent States Related Links Entity Name Attributes Processes (Services) Objects and Entities are static.

28. © Ramesh Jain Modeling a Domain Event Graphs: Capture relationships among events and entities. Event Transitions: Conditions and probabilities of transitions.

29. © Ramesh Jain Experience: Direct Observation of or Participation in Events as a basis of knowledge

30. © Ramesh Jain Webster definitions: Events: something that happens Data: factual information (as measurements or statistics) used as a basis for reasoning, discussion, or calculation Experience: direct observation of or participation in events as a basis of knowledge Insight: the act or result of apprehending the inner nature of things or of seeing intuitively Knowledge: the fact or condition of knowing something with familiarity gained through experience or association Information: the communication or reception of knowledge or intelligence

31. © Ramesh Jain Objectives Experiential environments are the next major technology inflexion point. Unified indexing of assimilated data is essential to implement experience-centric, rather than current information-centric, systems. Semantics is ubiquitous and emerges with symbiotic interactions among a user and the system.

32. © Ramesh Jain Information Integration: 2 User Integrated Database