1. 1 © Franz J. Kurfess Knowledge Interaction Franz J. Kurfess Cal Poly SLO Computer Science Department

2. 2 © Franz J. Kurfess Knowledge-Centric Interaction

3. Franz Kurfess: Knowledge Interaction Some of the material in these slides was developed for a lecture series sponsored by the European Community under the BPD program with Vilnius University as host institution Acknowledgements

4. 4 © Franz J. Kurfess Acknowledgements

5. 5 © Franz J. Kurfess Use and Distribution of these Slides ❖ These slides are primarily intended for the students in classes I teach. In some cases, I only make PDF versions publicly available. If you would like to get a copy of the originals (Apple KeyNote or Microsoft PowerPoint), please contact me via email at fkurfess@calpoly.edu. I hereby grant permission to use them in educational settings. If you do so, it would be nice to send me an email about it. If you’re considering using them in a commercial environment, please contact me first. fkurfess@calpoly.edu

6. 6 © Franz J. Kurfess Overview Knowledge Interaction ❖ Motivation ❖ Objectives ❖ Interactive Aspects of Knowledge  Different Perspectives  Interactive Organization  Interactive Search and Retrieval  Interactive Presentation and Visualization  Modeling and Simulation ❖ Examples ❖ Important Concepts and Terms ❖ Chapter Summary

7. 7 © Franz J. Kurfess Logistics

8. 8 © Franz J. Kurfess Preliminaries

9. 9 © Franz J. Kurfess Bridge-In ❖ How do you interact with the knowledge that you’re creating or using?  brain  paper  computer

10. 10 © Franz J. Kurfess Motivation and Objectives

11. 11 © Franz J. Kurfess Motivation

12. 12 © Franz J. Kurfess Objectives

13. 13 Interactive Aspects of Knowledge Different Perspectives Interactive Organization Interactive Search and Retrieval Interactive Presentation and Visualization Modeling and Simulation Evolution of Knowledge Knowledge Interaction Methods

14. 14 © Franz J. Kurfess Knowledge Perspectives ❖ simplistic assumption  there exists a coherent, well-organized body of knowledge for the domain under consideration ❖ the view of that knowledge may vary  role of the viewer  e.g. designer/developer vs. end user  presentation and viewing method and technology  color vs. black and white  text vs. graphical / auditory  purpose and task  look up facts  verify consistency  individual vs. collective view

15. 15 © Franz J. Kurfess Interactive Organization of Knowledge ❖ user/viewer takes an active role in the arrangement of knowledge  modification of categories and relations  creation of new instances  modification of content  resolution of inconsistencies ❖ examples:  Wikis  folksonomies, tagging systems  concept maps  ontologies

16. 16 © Franz J. Kurfess Knowledge Organization Approaches ❖ domain-centric  body of knowledge reflects structure and contents that are (more or less) agreed upon by the community ❖ content-oriented  the structure is derived from the content  e.g. Linnaeus’ taxonomy ❖ activity-oriented  the organization of knowledge is targeted for specific activities  e.g. instruction manuals, maintenance and repair documents ❖ individualistic  the organization and content are shaped by the views and preferences of an individual  computer directory structure, mail folders, file cabinets ❖ organization-centric  an organization has guidelines or standards for structure and content

17. 17 Interactive Search and Retrieval Incremental Search Faceted Search

18. 18 © Franz J. Kurfess Incremental Search and Retrieval ❖ query revision and reformulation  query restriction  additional keywords  query expansion  fewer keywords, synonyms  expanded search  eg. with logical operators ❖ improved search results through user feedback  relevance feedback has been investigated in Information Retrieval  not widely used in popular search engines

19. 19 © Franz J. Kurfess Faceted Search ❖ fine-tuning of results by filtering based on properties ❖ Example: Flight Search  more “information” than “knowledge”

20. 20 © Franz J. Kurfess Interactive Presentation and Visualization ❖ presentation mode  the user can select the preferred mode  textual, visual, auditory,... ❖ presentation adjustment  the user modifies parameters of the presentation  zooming, focus selection, perspective for 3D views,... ❖ visual browsing  exploration of material by following structural hints  hyper-links  special viewing modes such as thumbnail images  in contrast to searching by keywords or features

21. 21 © Franz J. Kurfess Interactive Presentation ❖ advantages  user is in control  allows exploration of knowledge repositories without a formulated query  visual features can be scanned easily and quickly ❖ problems  visual appearance of the arrangement can change  especially with automatic placement and arrangement  limited by available screen space  emphasis on easily visible aspects

22. 22 © Franz J. Kurfess Modeling and Simulation ❖ knowledge is captured and presented through models instead of descriptions  models  analytic approach  often abstract, formalized specification of entities  simulations  synthetic approach  implemented instances of models ❖ often capture dynamic aspects of systems  time, movement, shape change, processes, … ❖ often incorporate interactive aspects  educational, training, entertainment

23. 23 © Franz J. Kurfess Modeling and Simulation ❖ see also chapters on  Knowledge Representation  Knowledge Organization  Embodiment of Knowledge

24. 24 © Franz J. Kurfess Evolution of Knowledge ❖ content and structure of knowledge often change over time  most frequently through addition of new knowledge  consequences  inaccuracies  mismatch between the knowledge and the real world  often occurs as knowledge becomes obsolete  inconsistencies  conflicts between different pieces of knowledge  inadequate organization  the original structure of the knowledge is insufficient

25. 25 Knowledge Interaction Examples “Thought Control” interaction between computers and humans via brain signals (Brain-Computer Interfaces) Education and Training stepwise construction of mental models and knowledge spaces via computer interaction Complex Design development of complicated models and objects e.g. devices, machines, buildings, chemical compounds exploratory aspects designers enhance their knowledge through the activity

26. 26 © Franz J. Kurfess Thought Control ❖ sensors measure the activities in the brain  electrical  chemical ❖ actuators inject signals into the brain  electrical  chemical http://www.nature.com/news/2004/041013/images/brainchip.jpg

27. 27 © Franz J. Kurfess Thought Control Precepts ❖ activities are assumed to be related to thoughts ❖ technology allows the identification of areas related to certain types of thoughts  words, images ❖ limitations  practical and ethical considerations  most effective methods are invasive  brain implants  side effects  temporal and spatial resolution of sensors

28. 28 © Franz J. Kurfess Thought Control Devices ❖ invasive devices  implanted in the brain ❖ electrical field sensors  electro-encephalograms ❖ magnetic field sensors  MRI, fMRI http://technabob.com/blog/wp-content/uploads/2007/09/head_spa.jpg

29. 29 © Franz J. Kurfess Emotiv Epoc Headset  Sensors respond to the electrical impulses behind different thoughts;  enabling a user's brain to influence gameplay directly  Conscious thoughts, facial expressions, and non-conscious emotions can all be detected  Gyroscope enables a cursor or camera to be controlled by head movements  The headset uses wi-fi to connect to a computer http://news.bbc.co.uk/2/hi/technology/7254078.stm

30. 30 Earlier Attempts... Clockwork Orange (1971) BrainChip (2004) G-Tec g.EEGCap (2007)

31. 31 © Franz J. Kurfess Clockwork Orange (1971) A Clockwork Orange, Stanley Kubrick, 1971, film. http://www.pbs.org/wgbh/cultureshock/flashpoints/theater/clockworkorange_big.html

32. 32 © Franz J. Kurfess Igor Smirnov’s Device ❖ excerpt from Rumor Mill News Agency http://www.rumormillnews.com/def.htm http://www.rumormillnews.com/def.htm  Web site seems abandoned  I didn’t check the reports listed  an overview of related TV documentaries is at http://www.bibliotecapleyades.net/sociopolitica/ esp_sociopol_mindcon20.htm http://www.bibliotecapleyades.net/sociopolitica/ esp_sociopol_mindcon20.htm In the years 1993 and 1994 American weeklies DEFENSE ELECTRONICS (Defense Electronics, July 1993, DOD, Intel Agencies Look at Russian Mind- Control Technology, Claims FBI Considered Testing on Koresh), NEWSWEEK (Newsweek, February 7, 1994, Soon Phasers on Stun) and VILLAGE VOICE (Village Voice, March 8, 1994, Mind Control in Waco) published the information that Igor Smirnov from Moscow Academy of Medicine demonstrated for the U.S. secret services and FBI experts a device which was capable to subliminally implant thoughts in peoples minds and in this way control their actions....

33. 33 © Franz J. Kurfess Brain Chip (2004) ❖ brain implant with 100 electrodes ❖ used in research for people with severe disabilities  quadriplegics ❖ allows patients to use a computer ❖ requires major efforts to use http://www.nature.com/news/2004/041011/full/news041011-9.html http://www.wireheading.com/misc/implant.html

34. 34 http://www.mathworks.com/company/newsletters/news_notes/jan06/images/brain_figureB_wl.gif © Franz J. Kurfess Brain Control Block Diagram (2006) ❖ example of a MATLAB application http://www.mathworks.com/compa ny/newsletters/news_notes/jan06/b rain.html http://www.mathworks.com/compa ny/newsletters/news_notes/jan06/b rain.html ❖ used for brain research, applications  music composition ❖ see also EEGCapEEGCap 34 http://www.mathworks.com/company/newsletters/news_notes/jan06/images/brain_bci.gif

35. 35 © Franz J. Kurfess G-Tec g.EEGCap (2007) ❖ Cap to perform EEGs http://www.gtec.at/products/Accessories/gEEGcap.htm http://www.engadget.com/2007/03/16/g-tecs-thought-control-hat/

36. 36 © Franz J. Kurfess NeuroSky Thought Control System (2007) ❖ head-mounted brainwave sensor  currently only a research prototype  measures baseline brainwave activity in the brain  identifies states of “calmness”  probably similar to relaxation devices http://blog.wired.com/photos/uncategorized/2007/10/24/2.jpg

37. 37 © Franz J. Kurfess Ausklang

38. 38 © Franz J. Kurfess Post-Test

39. 39 © Franz J. Kurfess Evaluation ❖ Criteria

40. 40 © Franz J. Kurfess KP/KM Activity ❖ select a domain that requires significant human involvement for dealing with knowledge ❖ identify at least two candidates for  knowledge representation  reasoning ❖ evaluate their suitability  human perspective  understandable and usable for humans  computational perspective  storage, processing

41. 41 © Franz J. Kurfess KP/KM Activity Outcomes 2007 ❖ Images with Metadata ❖ Extracting contact information from text ❖ Qualitative and quantitative knowledge about cheese making ❖ Visualization of astronomy data ❖ Surveillance/security KM ❖ Marketing ❖ Face recognition ❖ Visual marketing

42. 42 © Franz J. Kurfess Important Concepts and Terms ❖ automated reasoning ❖ belief network ❖ cognitive science ❖ computer science ❖ deduction ❖ frame ❖ human problem solving ❖ inference ❖ intelligence ❖ knowledge acquisition ❖ knowledge representation ❖ linguistics ❖ logic ❖ machine learning ❖ natural language ❖ ontology ❖ ontological commitment ❖ predicate logic ❖ probabilistic reasoning ❖ propositional logic ❖ psychology ❖ rational agent ❖ rationality ❖ reasoning ❖ rule-based system ❖ semantic network ❖ surrogate ❖ taxonomy ❖ Turing machine

43. 43 © Franz J. Kurfess Summary Knowledge Interaction

44. 44 © Franz J. Kurfess