1. Franz Kurfess: Knowledge Retrieval Cal Poly SLO Computer Science Department Franz J. Kurfess Knowledge Interaction

2. Franz Kurfess: Knowledge Interaction Computer Science Department California Polytechnic State University San Luis Obispo, CA, U.S.A. Franz J. Kurfess Knowledge-Centric Interaction

3. Some of the material in these slides was developed for a lecture series sponsored by the European Community under the BPD program with Vilnius University as host institution Acknowledgements

4. Franz Kurfess: Knowledge Interaction This lecture series has been sponsored by the European Community under the BPD program with Vilnius University as host institution Acknowledgements

5. 5 Franz Kurfess: Internet2 and Education Use and Distribution of these Slides These slides are primarily intended for the students in classes I teach. In some cases, I only make PDF versions publicly available. If you would like to get a copy of the originals (Apple KeyNote or Microsoft PowerPoint), please contact me via email at fkurfess@calpoly.edu. I hereby grant permission to use them in educational settings. If you do so, it would be nice to send me an email about it. If you’re considering using them in a commercial environment, please contact me first. fkurfess@calpoly.edu 5

6. 6 Franz Kurfess: Knowledge Interaction Overview Knowledge Interaction ❖ Motivation ❖ Objectives ❖ Interactive Aspects of Knowledge ❖ Different Perspectives ❖ Interactive Organization ❖ Interactive Search and Retrieval ❖ Modeling and Simulation ❖ Important Concepts and Terms ❖ Chapter Summary 6

7. 7 Franz Kurfess: Knowledge Interaction Logistics 7

8. 8 Franz Kurfess: Knowledge Interaction Preliminaries 8

9. 9 Franz Kurfess: Knowledge Interaction Bridge-In ❖ How do you interact with the knowledge that you’re creating or using? ❖ brain ❖ paper ❖ computer 9

10. 10 Franz Kurfess: Knowledge Interaction Motivation and Objectives 10

11. 11 Franz Kurfess: Knowledge Interaction Motivation 11

12. 12 Franz Kurfess: Knowledge Interaction Objectives 12

13. 13 Franz Kurfess: Knowledge Interaction Interactive Aspects of Knowledge ❖ Different Perspectives ❖ Interactive Organization ❖ Interactive Search and Retrieval ❖ Interactive Presentation and Visualization ❖ Modeling and Simulation ❖ Evolution of Knowledge ❖ Knowledge Interaction Methods 13

14. 14 Franz Kurfess: Knowledge Interaction Knowledge Perspectives ❖ simplistic assumption ❖ there exists a coherent, well-organized body of knowledge for the domain under consideration ❖ the view of that knowledge may vary ❖ role of the viewer ❖ e.g. designer/developer vs. end user ❖ viewing method and technology ❖ color vs. black and white ❖ text vs. graphical / auditory ❖ purpose and task ❖ look up facts ❖ verify consistency ❖ individual vs. collective view 14

15. 15 Franz Kurfess: Knowledge Interaction Interactive Organization of Knowledge ❖ viewer takes an active role in the arrangement of knowledge ❖ modification of categories and relations ❖ creation of new instances ❖ modification of content ❖ resolution of inconsistencies ❖ examples: ❖ Wikis ❖ concept maps 15

16. 16 Franz Kurfess: Knowledge Interaction Knowledge Organization Approaches ❖ domain-centric ❖ body of knowledge reflects structure and contents that are (more or less) agreed upon by the community ❖ content-oriented ❖ the structure is derived from the content ❖ e.g. Linnaeus’ taxonomy ❖ activity-oriented ❖ the organization of knowledge is targeted for specific activities ❖ e.g. instruction manuals, maintenance and repair documents ❖ individualistic ❖ the organization and content are shaped by the views and preferences of an individual ❖ computer directory structure, mail folders, file cabinets ❖ organization-centric ❖ an organization has guidelines or standards for structure and content 16

17. 17 Franz Kurfess: Knowledge Interaction Interactive Search and Retrieval ❖ query revision and reformulation ❖ query restriction ❖ additional keywords ❖ query expansion ❖ fewer keywords, synonyms ❖ expanded search ❖ eg. with logical operators ❖ improved search results through user feedback ❖ relevance feedback has been investigated in Information Retrieval ❖ not widely used in popular search engines 17

18. 18 Franz Kurfess: Knowledge Interaction Interactive Presentation and Visualization ❖ presentation mode ❖ the user can select the preferred mode ❖ textual, visual, auditory,... ❖ presentation adjustment ❖ the user modifies parameters of the presentation ❖ zooming, focus selection,... ❖ visual browsing ❖ exploration of material by following structural hints ❖ hyperlinks 18

19. 19 Franz Kurfess: Knowledge Interaction Modeling and Simulation ❖ knowledge is captured and presented through models instead of descriptions ❖ models ❖ analytic approach ❖ often abstract, formalized specification of entities ❖ simulations ❖ synthetic approach ❖ implemented instances of models ❖ often capture dynamic aspects of systems ❖ time, movement, shape change, processes, … ❖ often incorporate interactive aspects ❖ educational, training, entertainment 19

20. 20 Franz Kurfess: Knowledge Interaction Evolution of Knowledge ❖ content and structure of knowledge often change over time ❖ most frequently through addition of new knowledge ❖ consequences ❖ inaccuracies ❖ mismatch between the knowledge and the real world ❖ often occurs as knowledge becomes obsolete ❖ inconsistencies ❖ conflicts between different pieces of knowledge ❖ inadequate organization ❖ the original structure of the knowledge is insufficient 20

21. 21 Franz Kurfess: Knowledge Interaction Knowledge Interaction Examples ❖ “Thought Control” ❖ interaction between computers and humans via brain signals (Brain-Computer Interfaces) ❖ Education and Training ❖ stepwise construction of mental models and knowledge spaces via computer interaction ❖ Complex Design ❖ development of complicated models and objects ❖ e.g. devices, machines, buildings, chemical compounds ❖ exploratory aspects ❖ designers enhance their knowledge through the activity 21

22. 22 Franz Kurfess: Knowledge Interaction Thought Control ❖ sensors measure the activities in the brain ❖ electrical ❖ chemical ❖ actuators inject signals into the brain ❖ electrical ❖ chemical 22 http://www.nature.com/news/2004/041013/images/brainchip.jpg

23. 23 Franz Kurfess: Knowledge Interaction Thought Control Precepts ❖ activities are assumed to be related to thoughts ❖ technology allows the identification of areas related to certain types of thoughts ❖ words, images ❖ limitations ❖ practical and ethical considerations ❖ most effective methods are invasive ❖ brain implants ❖ side effects ❖ temporal and spatial resolution of sensors 23

24. 24 Franz Kurfess: Knowledge Interaction Thought Control Devices ❖ invasive devices ❖ implanted in the brain ❖ electrical field sensors ❖ electro- encephalograms ❖ magnetic field sensors ❖ MRI, fMRI 24 http://technabob.com/blog/wp-content/uploads/2007/09/head_spa.jpg

25. 25 Franz Kurfess: Knowledge Interaction Emotiv Epoc Headset ❖ Sensors respond to the electrical impulses behind different thoughts; ❖ enabling a user's brain to influence gameplay directly ❖ Conscious thoughts, facial expressions, and non-conscious emotions can all be detected ❖ Gyroscope enables a cursor or camera to be controlled by head movements ❖ The headset uses wi-fi to connect to a computer 25 http://news.bbc.co.uk/2/hi/technology/7254078.stm

26. 26 Franz Kurfess: Knowledge Interaction Earlier Attempts... ❖ Clockwork Orange (1971) ❖ BrainChip (2004) ❖ G-Tec g.EEGCap (2007) 26

27. 27 Franz Kurfess: Knowledge Interaction Clockwork Orange (1971) 27 A Clockwork Orange, Stanley Kubrick, 1971, film. http://www.pbs.org/wgbh/cultureshock/flashpoints/theater/clockworkorange_big.html

28. 28 Franz Kurfess: Knowledge Interaction Igor Smirnov’s Device ❖ excerpt from Rumor Mill News Agency http://www.rumormillnews.com/def.htm http://www.rumormillnews.com/def.htm ❖ Web site seems abandoned ❖ I didn’t check the reports listed ❖ an overview of related TV documentaries is at http://www.bibliotecapleyades.net/sociopolitica/ esp_sociopol_mindcon20.htm http://www.bibliotecapleyades.net/sociopolitica/ esp_sociopol_mindcon20.htm 28 In the years 1993 and 1994 American weeklies DEFENSE ELECTRONICS (Defense Electronics, July 1993, DOD, Intel Agencies Look at Russian Mind-Control Technology, Claims FBI Considered Testing on Koresh), NEWSWEEK (Newsweek, February 7, 1994, Soon Phasers on Stun) and VILLAGE VOICE (Village Voice, March 8, 1994, Mind Control in Waco) published the information that Igor Smirnov from Moscow Academy of Medicine demonstrated for the U.S. secret services and FBI experts a device which was capable to subliminally implant thoughts in peoples minds and in this way control their actions....

29. 29 Franz Kurfess: Knowledge Interaction Brain Chip (2004) ❖ brain implant with 100 electrodes ❖ used in research for people with severe disabilities ❖ quadriplegics ❖ allows patients to use a computer ❖ requires major efforts to use 29 http://www.nature.com/news/2004/041011/full/news041011-9.html http://www.wireheading.com/misc/implant.html

30. 30 http://www.mathworks.com/company/newsletters/news_notes/jan06/images/brain_figureB_wl.gif Franz Kurfess: Knowledge Interaction Brain Control Block Diagram (2006) ❖ example of a MATLAB application http://www.mathworks.com/company/n ewsletters/news_notes/jan06/brain.ht ml http://www.mathworks.com/company/n ewsletters/news_notes/jan06/brain.ht ml ❖ used for brain research, applications ❖ music composition ❖ see also EEGCapEEGCap 30 http://www.mathworks.com/company/newsletters/news_notes/jan06/images/brain_bci.gif

31. 31 Franz Kurfess: Knowledge Interaction G-Tec g.EEGCap (2007) ❖ Cap to perform EEGs 31 http://www.gtec.at/products/Accessories/gEEGcap.htm http://www.engadget.com/2007/03/16/g-tecs-thought-control-hat/

32. 32 Franz Kurfess: Knowledge Interaction NeuroSky Thought Control System (2007) ❖ head-mounted brainwave sensor ❖ currently only a research prototype ❖ measures baseline brainwave activity in the brain ❖ identifies states of “calmness” ❖ probably similar to relaxation devices 32 http://blog.wired.com/photos/uncategorized/2007/10/24/2.jpg

33. 33 Franz Kurfess: Knowledge Interaction Ausklang 33

34. 34 Franz Kurfess: Knowledge Interaction Post-Test 34

35. 35 Franz Kurfess: Knowledge Interaction Evaluation ❖ Criteria 35

36. 36 Franz Kurfess: Knowledge Interaction KP/KM Activity ❖ select a domain that requires significant human involvement for dealing with knowledge ❖ identify at least two candidates for ❖ knowledge representation ❖ reasoning ❖ evaluate their suitability ❖ human perspective ❖ understandable and usable for humans ❖ computational perspective ❖ storage, processing 36

37. 37 Franz Kurfess: Knowledge Interaction KP/KM Activity Outcomes 2007 ❖ Images with Metadata ❖ Extracting contact information from text ❖ Qualitative and quantitative knowledge about cheese making ❖ Visualization of astronomy data ❖ Surveillance/security KM ❖ Marketing ❖ Face recognition ❖ Visual marketing 37

38. 38 Franz Kurfess: Knowledge Interaction Important Concepts and Terms 38 ❖ automated reasoning ❖ belief network ❖ cognitive science ❖ computer science ❖ deduction ❖ frame ❖ human problem solving ❖ inference ❖ intelligence ❖ knowledge acquisition ❖ knowledge representation ❖ linguistics ❖ logic ❖ machine learning ❖ natural language ❖ ontology ❖ ontological commitment ❖ predicate logic ❖ probabilistic reasoning ❖ propositional logic ❖ psychology ❖ rational agent ❖ rationality ❖ reasoning ❖ rule-based system ❖ semantic network ❖ surrogate ❖ taxonomy ❖ Turing machine

39. 39 Franz Kurfess: Knowledge Interaction Summary Knowledge Interaction 39

40. 40 Franz Kurfess: Knowledge Interaction 40