1. Week 8 MSE614 – SP 08 Ileana Costea

2. HW Questions on KA Due today, Week 8 Assigned last session, Week 7 A few verbal questions (see Transparency)

3. HW Question #1 What is meta-knowledge? Answer: K about K, or The system’s K about how it reasons e.g., how to use K in specific situations; how to determine which K is relevant when the K is insufficient

4. Meta-knowledge (ctnd.) Meta-K allows the system to examine the operation of the descriptive (declarative) and procedural K Explanation in an ES can be viewed as Meta-K In the future Meta-K will allow ES more: –Create the rationale behind individual rules –by reasoning from first principles –Tailor explanations to fit the audience –Able to change their internal structure through: –Rule correction –Reorganization of KB –System reconfiguration

5. HW Question #2 Give four reasons why KA is difficult. Representation mismatch between the human Expert and the program underdevelopment Large number of participants Transfer via a machine Difficulty of experts to describe their K More … next slide

6. HW Question #2 (Ctnd.) Give four reasons why KA is difficult. Other reasons, see slide. Overcoming difficulties: Research on KA to reduce mismatch: Develop systems able to accept advice KA to converse with expert in natural language Simplify the syntax of the rules so that an expert can build a system without training

7. HW Question #3 Describe the process of Protocol Analysis. (Ctnd.) The expert is asked to perform a real task. Then he/she is asked to verbalize (think aloud) the thought process. A record (protocol) of all the details is made. The records are transcribed.

8. HW Question #3 Describe the process of Protocol Analysis. (Ctnd.) Required Web search: http://www.psy.fsu.edu/faculty/ericsson/ericsson.proto.th nk.htmlhttp://www.psy.fsu.edu/faculty/ericsson/ericsson.proto.th nk.html Protocol analysis and Verbal Reports on Thinking An updated and extracted version from Ericsson (2002) http://www.epistemics.co.uk/Notes/176-0-0.htm Protocol Analysis Techniques Among others discusses Repertory Grid Technique http://en.wikipedia.org/wiki/Protocol_analysis

9. Protocol Analysis http://en.wikipedia.org/wiki/Protocol_analysis (from Wikipedia, the free encyclopedia) Protocol analysis is a psychological research method that elicits verbal reports from research participants. Protocol analysis is used to study thinking in cognitive psychology, cognitive science, and behavior analysis. It has found further application in the design of surveys and interviews, usability testing, and educational psychology.psychologicalcognitive psychologycognitive scienceusability testingeducational psychology Usability testing is a technique used to evaluate a product by testing it on users. Cognitive psychology is a school of thought in psychology that examines internal mental processes such as problem solving, memory, and language.

10. Cognitive science (from Wikipidia) the scientific study either of mind or of intelligence. an interdisciplinary study drawing from relevant fields including psychology, philosophy, neuroscience, linguistics, anthropology, computer science, and biology. psychologyphilosophy neurosciencelinguistics anthropologycomputer sciencebiology term coined by Christopher Longuet-Higgins in his 1973 commentary on the Lighthill report, which concerned the then- current state of Artificial Intelligence research. In the same decade, the journal Cognitive Science and the Cognitive Science Society began.Christopher Longuet-HigginsLighthill report

11. Protocol Analysis (Ctnd.) in AI – KA it is a method of KA it is one of the “tracking methods” it is the most common method of formal tracking. (See next slide for “tracking”)

12. Tracking Methods (Protocol Analysis – Ctnd.) Process tracking: a set of techniques that attempt to track the reasoning process of an expert. Popular method among cognitive psychologists who: interested in discovering the expert’s “train of thought” while he/she reaches a conclusion Tracking methods are: informal and formal

13. Protocol Analysis (Ctnd.) Particularly a set of techniques known as verbal protocol analysis, a common method by which KEngineer acquires detailed K from the expert

14. Protocol Analysis (Ctnd.) A protocol = a record or documentation of the expert’s step-by-step info. processing (IP) and decision-making (DM) behavior Protocol Analysis Is similar to interviewing but more formal and systematic The expert is asked by the KE to perform a real task and verbalize his/her thought process Expert is asked to “think aloud” while performing the task or solving a problem under observation A recording made while expert “thinks aloud” Recording describes every aspect of the IP and DM behavior The recording becomes a record, or protocol, of the expert’s ongoing behavior

15. Protocol Analysis (ctnd.) Recording is: –transcribed by the KE for further analysis ( e.g., to deduce the decision process) –Coded by the KE

16. Protocol Analysis (ctnd.) In contrast with interactive interviewing methods, PA involves mainly a one-way communication. KE prepares the scenario and plans the process during the session the Expert does most of the talking as he/she interacts with the data to solve problem KE listens and records the process Later KE must: analyze, interpret, and structure the protocol (or record) into Knowledge Representation for a review by the Expert.

17. Protocol Analysis (ctnd.) for Procedure of PA & Advantages and Limitations of PA  see Transparencies

18. HW Question #4 List the major difficulties of KA from multiple experts. Different experts use different methods of problem solving, yet all may be correct. It is difficult to reconcile these different methods. It is difficult to get all the experts together at the same time and place.

19. List the major difficulties of KA from multiple experts. (Ctnd.) The four possible scenarios to deal with multiple experts: –individual experts –primary and secondary experts –small groups –panels

20. List the major difficulties of KA from multiple experts. (Ctnd.) Major methods of dealing with multiple experts: Consensus methods Reach consensus by using group dynamics or other methodologies Analytical approaches: figure the average estimate of the group Selection of an appropriate line of reasoning for each occasion – don’t mix Automation of the process automatic decision by the computer Blackboard system divide the problem into sub-domains and use one expert for each sub-domain

21. HW Question #4 Define evaluation, validation and verification of K.

22. AAAI http://www.aaai.org/home.html http://www.aaai.org/home.html Association for the Advancement of Artificial Intelligence (AAAI) (formerly the American Association for Artificial Intelligence) is a nonprofit scientific society devoted to advancing the scientific understanding of: mechanisms underlying thought & intelligent behavior, and their embodiment in machines

23. AI Applications http://www.aaai.org/AITopics/pmwiki/pmwi ki.php/AITopics/SiteMap a whole list of links of AI applications http://www.aaai.org/AITopics/pmwiki/pmwi ki.php/AITopics/AINews AI in the News