Chapter 1: Introduction to Expert Systems

Contents Introduction Knowledge Representing General Methods of Inference

Objectives Learn the meaning of an expert system Understand the problem domain and knowledge domain Learn the advantages of an expert system Understand the stages in the development of an expert system Examine the general characteristics of an expert system Examine earlier expert systems which have given rise to today’s knowledge-based systems Explore the applications of expert systems in use today Examine the structure of a rule-based expert system

What is an expert system? “An expert system is a computer system that emulates, or acts in all respects, with the decision-making capabilities of a human expert.”

Fig 1.1 Areas of Artificial Intelligence

Expert system technology may include: Special expert system languages – CLIPS Programs Hardware designed to facilitate the implementation of those systems Expert System Main Components: Knowledge base – obtainable from books, magazines, knowledgeable persons, etc. Inference engine – draws conclusions from the knowledge base

Figure 1.2 Basic Functions of Expert Systems

Problem Domain vs. Knowledge Domain An expert’s knowledge is specific to one problem domain – medicine, finance, science, engineering, etc. The expert’s knowledge about solving specific problems is called the knowledge domain. The problem domain is always a superset of the knowledge domain.

Figure 1.3 Problem and Knowledge Domain Relationship

Advantages of Expert Systems Increased availability Reduced cost Reduced danger Performance Multiple expertise Increased reliability Explanation Fast response Steady, unemotional, and complete responses at all times Intelligent tutor Intelligent database

2. Knowledge Representing The knowledge of an expert system can be represented in a number of ways, including IF-THEN rules: IF you are hungry THEN eat

Knowledge Engineering The process of building an expert system: The knowledge engineer establishes a dialog with the human expert to elicit knowledge. The knowledge engineer codes the knowledge explicitly in the knowledge base. The expert evaluates the expert system and gives a critique to the knowledge engineer.

Fig 1.4 Development of an Expert System

The Role of AI An algorithm is an ideal solution guaranteed to yield a solution in a finite amount of time. When an algorithm is not available or is insufficient, we rely on artificial intelligence (AI). Expert system relies on inference – we accept a “reasonable solution.”

Uncertainty Both human experts and expert systems must be able to deal with uncertainty. It is easier to program expert systems with shallow knowledge than with deep knowledge. Shallow knowledge – based on empirical and heuristic knowledge. Deep knowledge – based on basic structure, function, and behavior of objects.

Limitations of Expert Systems Typical expert systems cannot generalize through analogy to reason about new situations in the way people can. A knowledge acquisition bottleneck results from the time-consuming and labor intensive task of building an expert system. Early Expert Systems DENDRAL – used in chemical mass spectroscopy to identify chemical constituents MYCIN – medical diagnosis of illness DIPMETER – geological data analysis for oil PROSPECTOR – geological data analysis for minerals XCON/R1 – configuring computer systems

Table 1.1 Broad Classes of Expert Systems

Problems with Algorithmic Solutions Conventional computer programs generally solve problems having algorithmic solutions. Algorithmic languages include C, Java, and C#. Classical AI languages include LISP and PROLOG.

Considerations for Building Expert Systems Can the problem be solved effectively by conventional programming? Is there a need and a desire for an expert system? Is there at least one human expert who is willing to cooperate? Can the expert explain the knowledge to the knowledge engineer and he can understand it? Is the problem-solving knowledge mainly heuristic and uncertain?

Languages, Shells, and Tools Expert system languages are post-third generation. Procedural languages (e.g., C) focus on techniques to represent data. More modern languages (e.g., Java) focus on data abstraction. Expert system languages (e.g. CLIPS) focus on ways to represent knowledge.

Expert systems & conventional programs I

Expert systems & conventional programs II

Expert systems & conventional programs III

Elements of an Expert System User interface – mechanism by which user and system communicate. Exploration facility – explains reasoning of expert system to user. Working memory – global database of facts used by rules. Inference engine – makes inferences deciding which rules are satisfied and prioritizing. Agenda – a prioritized list of rules created by the inference engine, whose patterns are satisfied by facts or objects in working memory. Knowledge acquisition facility – automatic way for the user to enter knowledge in the system by passing the explicit coding by knowledge engineer. Knowledge Base – includes the rules of the expert system

Production Rules Knowledge base is also called production memory. Production rules can be expressed in IF-THEN pseudo code format. In rule-based systems, the inference engine determines which rule antecedents are satisfied by the facts.

Figure 1.5 Structure of a Rule-Based Expert System

Rule-Based ES

Example Rules

3. Inference Engine Cycle

Foundation of Expert Systems

General Methods of Inference Forward chaining (data-driven)– reasoning from facts to the conclusions resulting from those facts – best for prognosis, monitoring, and control. Examples: CLIPS, OPS5 Backward chaining (query/Goal driven)– reasoning in reverse from a hypothesis, a potential conclusion to be proved to the facts that support the hypothesis – best for diagnosis problems. Examples: MYCIN

Production Systems Rule-based expert systems – most popular type today. Knowledge is represented as multiple rules that specify what should/not be concluded from different situations. Forward chaining – start w/facts and use rules do draw conclusions/take actions. Backward chaining – start w/hypothesis and look for rules that allow hypothesis to be proven true.

Post Production System Basic idea – any mathematical / logical system is simply a set of rules specifying how to change one string of symbols into another string of symbols. these rules are also known as rewrite rules simple syntactic string manipulation no understanding or interpretation is required\also used to define grammars of languages e.g BNF grammars of programming languages. Basic limitation – lack of control mechanism to guide the application of the rules.

Reasoning Under Uncertainty Chapter 2: Reasoning Under Uncertainty

Contents Introduction Dealing with Uncertainty Classical Probability

Objectives Learn the meaning of uncertainty and explore some theories designed to deal with it Find out what types of errors can be attributed to uncertainty and induction Learn about classical probability, experimental, and subjective probability, and conditional probability Explore hypothetical reasoning and backward induction Examine temporal reasoning and Markov chains Define odds of belief, sufficiency, and necessity Determine the role of uncertainty in inference chains Explore the implications of combining evidence Look at the role of inference nets in expert systems and see how probabilities are propagated

How to Expert Systems Deal with Uncertainty? Expert systems provide an advantage when dealing with uncertainty as compared to decision trees. With decision trees, all the facts must be known to arrive at an outcome. Probability theory is devoted to dealing with theories of uncertainty. There are many theories of probability – each with advantages and disadvantages.

What is Uncertainty? Uncertainty is essentially lack of information to formulate a decision. Uncertainty may result in making poor or bad decisions. As living creatures, we are accustomed to dealing with uncertainty – that’s how we survive. Dealing with uncertainty requires reasoning under uncertainty along with possessing a lot of common sense.

2. Dealing with Uncertainty Deductive reasoning – deals with exact facts and exact conclusions Inductive reasoning – not as strong as deductive – premises support the conclusion but do not guarantee it. There are a number of methods to pick the best solution in light of uncertainty. When dealing with uncertainty, we may have to settle for just a good solution.

Theories to Deal with Uncertainty Bayesian Probability Hartley Theory Shannon Theory Dempster-Shafer Theory Markov Models Zadeh’s Fuzzy Theory

Errors Related to Hypothesis Many types of errors contribute to uncertainty. Type I Error – accepting a hypothesis when it is not true – False Positive. Type II Error – Rejecting a hypothesis (theory) when it is true – False Negative

Errors Related to Measurement Errors of precision – how well the truth is known Errors of accuracy – whether something is true or not Unreliability stems from faulty measurement of data – results in erratic data. Random fluctuations – termed random error Systematic errors result from bias

Errors in Induction Where deduction proceeds from general to specific, induction proceeds from specific to general. Inductive arguments can never be proven correct (except in mathematical induction). Expert systems may consist of both deductive and inductive rules based on heuristic information. When rules are based on heuristics, there will be uncertainty.

Figure 2.1Deductive and Inductive Reasoning about Populations and Samples

Figure 2.2 Types of Errors

Table 2.1 Examples of Common Types of Errors

3. Classical Probability First proposed by Pascal and Fermat in 1654 Also called a priori probability because it deals with ideal games or systems: Assumes all possible events are known Each event is equally likely to happen Fundamental theorem for classical probability is P = W / N, where W is the number of wins and N is the number of equally possible events.

Deterministic &Nondeterministic Systems When repeated trials give the exact same results, the system is deterministic. Otherwise, the system is nondeterministic. Nondeterministic does not necessarily mean random – could just be more than one way to meet one of the goals given the same input.

Three Axioms of Formal Theory of Probability

Experimental and Subjective Probabilities Experimental probability defines the probability of an event, as the limit of a frequency distribution: Subjective probability deals with events that are not reproducible and have no historical basis on which to extrapolate.

Compound Probabilities Compound probabilities can be expressed by: S is the sample space and A and B are events. Independent events are events that do not affect each other. For pair wise independent events,

Additive Law

Conditional Probabilities The probability of an event A occurring, given that event B has already occurred is called conditional probability:

Figure 2.3 Sample Space of Intersecting Events

Bayes’ Theorem This is the inverse of conditional probability. Find the probability of an earlier event given that a later one occurred.

Hypothetical Reasoning Backward Induction Bayes’ Theorem is commonly used for decision tree analysis of business and social sciences. PROSPECTOR (expert system) achieved great fame as the first expert system to discover a valuable molybdenum deposit worth $100,000,000.

Temporal Reasoning Reasoning about events that depend on time Expert systems designed to do temporal reasoning to explore multiple hypotheses in real time are difficult to build. One approach to temporal reasoning is with probabilities – a system moving from one state to another over time. The process is stochastic if it is probabilistic.

Reasoning about events that depend on time Transition matrix – represents the probabilities that the system in one state will move to another. State matrix – depicts the probabilities that the system is in any certain state. One can show whether the states converge on a matrix called the steady-state matrix – a time of equilibrium

Markov Chain Characteristics The process has a finite number of possible states. The process can be in one and only one state at any one time. The process moves or steps successively from one state to another over time. The probability of a move depends only on the immediately preceding state.

Figure 2.4 State Diagram Interpretation of a Transition Matrix

The Odds of Belief To make expert systems work for use, we must expand the scope of events to deal with propositions. Rather than interpreting conditional probabilities P(A!B) in the classical sense, we interpret it to mean the degree of belief that A is true, given B. We talk about the likelihood of A, based on some evidence B. This can be interpreted in terms of odds.

Sufficiency and Necessity The likelihood of sufficiency, LS, is calculated as: The likelihood of necessity is defined as:

Table 2.2 Relationship Among Likelihood Ratio, Hypothesis, and Evidence

Table 2.3 Relationship Among Likelihood of Necessity, Hypothesis, and Evidence

Uncertainty in Inference Chains Uncertainty may be present in rules, evidence used by rules, or both. One way of correcting uncertainty is to assume that P(H|e) is a piecewise linear function.

Figure 2.5 Intersection of H and e

Figure 2.6 Piecewise Linear Interpolation Function for Partial Evidence in PROSPECTOR

Combination of Evidence The simplest type of rule is of the form: IF E THEN H where E is a single piece of known evidence from which we can conclude that H is true. Not all rules may be this simple – compensation for uncertainty may be necessary. As the number of pieces of evidence increases, it becomes impossible to determine all the joint and prior probabilities or likelihoods. If the antecedent is a logical combination of evidence, then fuzzy logic and negation rules can be used to combine evidence.

Types of Belief Possible – no matter how remote, the hypothesis cannot be ruled out. Probable – there is some evidence favoring the hypothesis but not enough to prove it. Certain – evidence is logically true or false. Impossible – it is false. Plausible – more than a possibility exists.

Figure 2.7 Relative Meaning of Some Terms Used to Describe Evidence

Propagation of Probabilities The chapter examines the classic expert system PROSPECTOR to illustrate how concepts of probability are used in a real system. Inference nets like PROSPECTOR have a static knowledge structure. Common rule-based system is a dynamic knowledge structure.

Chapter 3 Fuzzy Logic

B. Inputs to fuzzy logic systems - fuzzification C. Fuzzy propositions A. Introduction B. Inputs to fuzzy logic systems - fuzzification C. Fuzzy propositions D. Fuzzy hedges E. Computing the results of a fuzzy proposition given an input F. The resulting action

Fuzzy Logic Objectives 1. To define the basic notions of fuzzy logic 2. To introduce the logical operations and relations on fuzzy sets 3. To learn how to obtain results of fuzzy logical operations 4. To apply what we learn to GIS

A. Introduction (figure from Earl Cox)

Introduction Steps (Earl Cox based on previous slide): 1. Input – vocabulary, fuzzification (creating fuzzy sets) 2. Fuzzy propositions – IF X is Y THEN Z (or Z is A) … there are four types of propositions 3. Hedges – very, extremely, somewhat, more, less 4. Combination and evaluation – computation of the results given the inputs 5. Action - defuzzification

Input – vocabulary, fuzzification (creating a fuzzy set) by using our previous methods of frequency, combination, experts/surveys (figure from Earl Cox)

Input (figure from Klir&Yuan)

Fuzzy Propositions – types 1 and 2 GENERAL FORMS 1. Unconditional and unqualified proposition: Q is P Example: Temperature(Q) is high(P) 2. Unconditional and qualified proposition: proposition(Q is P) is R Example: That Coimbra and Catania are beautiful is very true.

Fuzzy Proposition – type 1 and 2 (from Earl Cox)

Fuzzy Propositions – type 1 and 2 (from Earl Cox)

Fuzzy Propositions – type 3 3. Conditional and unqualified proposition: IF Q is P THEN R is S Example: If Robert is tall, then clothes are large. If car is slow, then gear is low.

Fuzzy Propositions – type 4 4. Conditional and qualified proposition: IF Q is P THEN R is S is T {proposition(IF Q is P THEN R is S )} is T

Fuzzy Hedges (from Earl Cox)

Fuzzy Hedges (from Earl Cox)

Illustrations of Fuzzy Propositions – Composition/Evaluation (from Klir&Yuan)

Illustrations of Fuzzy Propositions – Composition/Evaluation (Earl Cox)

Illustrations of Fuzzy Propositions – Composition/Evaluation (from Earl Cox)

Illustrations of Fuzzy Propositions Decomposition – Defuzzification/Action (from Earl Cox)

Defuzzification (from Earl Cox)

Defuzzification (from Earl Cox)

Chapter 4 Design of Expert Systems

Considerations when Building an Expert system We will describe the necessary prerequisites when building an expert system. The system should be a quality product. The development should be cost effective and timely. Designing of expert systems of part of a general field known as Knowledge Management (KM).

Selecting the Appropriate Problem We need to answer the questions, “Why are we building this expert system?”. Intellectual Property Agreement must be considered Clearly identify the problem Clearly identify the expert Clearly identify the users We need to know the payoff – money, efficiency, etc.

Selecting the Appropriate Problem What tools will be available to build the expert system? Check the Web for applications in existence Know the language necessary to create a semantic net of relationships on which the system will be based How much will the expert system cost? A function of people, resources, time, etc. How available is the knowledge?

Figure 4.1 Project Management Tasks

Stages in the Development How will the system be developed? This will depend on the resources provided Stages: Feasibility Study – see if the project is feasible Rapid Prototype – demonstrate ideas / impress Refined System – verification by knowledge engineers Field Testable – system tested by selected users

Stages in the Development Commercial quality system – validation / testing Maintenance and evolution – repair bugs, enhance capabilities

Other Considerations How will the system be delivered? Should be considered in earliest stages of development Integration with existing programs How will the system be maintained and evolve? Performance is dependent on knowledge / expertise Performance must be maintained New knowledge will be acquired Old knowledge will be modified

Figure 4.2 General Stages in the Development of an Expert System

Errors in Development Stages Expert’s knowledge may be erroneous, propagating errors throughout the entire development process. Formal procedures may be necessary to certify expert Technique panels can scrutinize expert’s knowledge Focus groups can also be used

Errors in the Development Stages Knowledge may not be properly communicated to knowledge engineer, or knowledge may be misinterpreted. Knowledge base may be corrupted by entering incorrect form of a rule or fact.

Errors in the Development Stages Inference engine errors may result from errors in pattern matching, conflict resolution, and execution of actions. Inference chain errors may be caused by erroneous knowledge, semantic errors, inference engine bugs, incorrect specifications of rule priorities, and strange interaction among rules. Limits of ignorance – a problem common to all previous stages

Figure 4.3 Major Errors in Expert Systems and Some Causes

Software Engineering & Expert Systems Expert systems are products like any other software product and require good standards for development. Expert systems may have serious responsibilities – life and death. High standards are a necessity and can be measured by “mean time between failures”.

Figure 4.4 Software Engineering Methodology

Expert System Life Cycle Begins with the initial concept of the software and ends with its retirement from use. Expert systems require more maintenance because they are based on knowledge that is: Heuristic Experiential A number of life cycle models have been developed.

Waterfall Model Each stage ends with a verification and validation activity to minimize any problems in that stage. Arrows go back and forth only one stage at a time. It is assumed that all information necessary for a stage is known.

Figure 4.5 Waterfall Model of the Software Life Cycle

Code-and-Fix Model Some code is written and then fixed when it does not work correctly. Usually the method of choice for new programming students in conventional and expert systems This eventually led to the do-it-twice concept where a prototype then a final system was built.

Incremental Model This is a refinement of the waterfall and top-down-approach. The idea is to develop software in increments of functional capability. Major increment – assistant  colleague  expert Minor increment – expertise w/in each level Micro increment – add/refining individual rules

Figure 4.6 Spiral Model Each circuit of the spiral adds some functional capability to the system.

Detailed Life Cycle Model Linear Model Planning Stage The purpose of this stage is to produce a formal work plan for the expert system development – documents to guide and evaluate the development.

Table 4.1 Planning Stage Tasks

Linear Model Knowledge Definition The objective of this stage is to define the knowledge requirements of the expert system, which consists of two main tasks: Knowledge source identification and selection Knowledge acquisition, analysis, and extraction

Table 4.2 Knowledge Source / Identification

Table 4.3 Knowledge Acquisition, Analysis, and Extraction Tasks

Linear Model Knowledge Design The objective is to produce the detailed design for an expert system and involves: Knowledge definition Detailed design

Table 4.4 Knowledge Definition Tasks

Table 4.5 Detailed Design of Knowledge Tasks

Linear Model Code and Checkout This begins the actual code implementation

Table 4.6 Code and Checkout Tasks

Linear Model Knowledge Verification The objective here is to determine the correctness, completeness, and consistency of the system. Formal tests Test Analysis

Table 6.7 Formal Test Tasks of Knowledge Verification Stage

Table 6.8 Test Analysis Tasks

Linear Model System Evaluation This stage is for summarizing what has been learned with recommendations for improvements and corrections.

Table 4.9 System Evaluation Stage Tasks

Figure 4.7 Linear Model of Expert System Development Life Cycle

Summary In this chapter, we have discussed a software engineering approach to the construction of an expert system. Principles about good interviewing techniques were mentioned. Now that expert systems are widely used, they must be quality products due to the sensitive nature of their applications.

Summary Factors to be considered in the design of expert systems include problem selection, cost, and payoff. Both managerial and technical aspects must be considered. By consistently following the outlined “life cycle” it should be possible to construct quality software.