© Franz J. Kurfess Expert System Design 1 CPE/CSC 481: Knowledge-Based Systems Dr. Franz J. Kurfess Computer Science Department Cal Poly

© Franz J. Kurfess Expert System Design 2 Usage of the Slides u these slides are intended for the students of my CPE/CSC 481 “Knowledge-Based Systems” class at Cal Poly SLO u if you want to use them outside of my class, please let me know u I usually put together a subset for each quarter as a “Custom Show” u to view these, go to “Slide Show => Custom Shows”, select the respective quarter, and click on “Show” u To print them, I suggest to use the “Handout” option u 4, 6, or 9 per page works fine u Black & White should be fine; there are few diagrams where color is important

© Franz J. Kurfess Expert System Design 3 Course Overview u Introduction u Knowledge Representation u Semantic Nets, Frames, Logic u Reasoning and Inference u Predicate Logic, Inference Methods, Resolution u Reasoning with Uncertainty u Probability, Bayesian Decision Making u Expert System Design u ES Life Cycle u CLIPS Overview u Concepts, Notation, Usage u Pattern Matching u Variables, Functions, Expressions, Constraints u Expert System Implementation u Salience, Rete Algorithm u Expert System Examples u Conclusions and Outlook

© Franz J. Kurfess Expert System Design 4 Overview Expert System Design u Motivation u Objectives u Chapter Introduction u Review of relevant concepts u Overview new topics u Terminology u ES Development Life Cycle u Feasibility Study u Rapid Prototype u Refined System u Field Testable u Commercial Quality u Maintenance and Evolution u Software Engineering and ES Design u Software Development Life Cycle u Linear Model ES Life Cycle u Planning u Knowledge Definition u Knowledge Design u Knowledge Verification u Important Concepts and Terms u Chapter Summary

© Franz J. Kurfess Expert System Design 5 Material [Awad 1996][Awad 1996] Material [Awad 1996][Awad 1996] u Chapter 5: Expert System Development Life Cycle u Chapter 15: Verification and Validation u Chapter 17: Implementing the Expert System u Chapter 18: Organizational and Managerial Impact

© Franz J. Kurfess Expert System Design 6 Material [Durkin 1994][Durkin 1994] Material [Durkin 1994][Durkin 1994] u Chapter: 8: Designing Backward-Chaining Rule- Based Systems u Chapter 10: Designing Forward-Chaining Rule- Based Systems u Chapter 15: Designing Frame-Based Expert Systems u Chapter 18: Knowledge Engineering

© Franz J. Kurfess Expert System Design 7 Material [Jackson 1999][Jackson 1999] Material [Jackson 1999][Jackson 1999] u Chapter 14, 15: Constructive Problem Solving u Chapter 16: Designing for Explanation

© Franz J. Kurfess Expert System Design 8 Material [Sommerville 2001][Sommerville 2001] Material [Sommerville 2001][Sommerville 2001] u Chapter 3: Software processes u waterfall model u evolutionary development v spiral model u formal methods u reuse-based methods u Chapter 8: Software prototyping u rapid prototyping techniques

© Franz J. Kurfess Expert System Design 9 Logistics u Introductions u Course Materials u textbooks (see below) u lecture notes u PowerPoint Slides will be available on my Web page u handouts u Web page u u Term Project u Lab and Homework Assignments u Exams u Grading

© Franz J. Kurfess Expert System Design 10 Bridge-In

© Franz J. Kurfess Expert System Design 11 Pre-Test

© Franz J. Kurfess Expert System Design 12 Motivation u reasons to study the concepts and methods in the chapter u main advantages u potential benefits u understanding of the concepts and methods u relationships to other topics in the same or related courses

© Franz J. Kurfess Expert System Design 13 Objectives u regurgitate u basic facts and concepts u understand u elementary methods u more advanced methods u scenarios and applications for those methods u important characteristics v differences between methods, advantages, disadvantages, performance, typical scenarios u evaluate u application of methods to scenarios or tasks u apply u methods to simple problems

© Franz J. Kurfess Expert System Design 14 ES Development Methods u commercial quality systems require a systematic development approach u ad hoc approaches may be suitable for research prototypes or personal use, but not for widely used or critical systems u some software engineering methods are suitable for the development of expert systems

© Franz J. Kurfess Expert System Design 15 Problem Selection u the development of an expert system should be based on a specific problem to be addressed by the system u it should be verified that expert systems are the right paradigm to solve that type of problem u not all problems are amenable to ES-based solutions u availability of resources for the development u experts/expertise u hardware/software u users u sponsors/funds

© Franz J. Kurfess Expert System Design 16 Project Management u activity planning u planning, scheduling, chronicling, analysis u product configuration management u product management u change management u resource management u need determination u acquisition resources u assignment of responsibilities u identification of critical resources

© Franz J. Kurfess Expert System Design 17 ES Development Stages u feasibility study u paper-based explanation of the main idea(s) u no implementation u rapid prototype u quick and dirty implementation of the main idea(s) u refined system u in-house verification by knowledge engineers, experts u field test u system tested by selected end users u commercial quality system u deployed to a large set of end users u maintenance and evolution u elimination of bugs u additional functionalities

© Franz J. Kurfess Expert System Design 18 Error Sources in ES Development u knowledge errors u semantic errors u syntax errors u inference engine errors u inference chain errors u limits of ignorance errors

© Franz J. Kurfess Expert System Design 19 Knowledge Errors u problem: knowledge provided by the expert is incorrect or incomplete u reflection of expert’s genuine belief u omission of important aspects u inadequate formulation of the knowledge by the expert u consequences u existing solution not found u wrong conclusions u remedy u validation and verification of the knowledge v may be expensive

© Franz J. Kurfess Expert System Design 20 Semantic Errors u problem: the meaning of knowledge is not properly communicated u knowledge engineer encodes rules that do not reflect what the domain expert stated u expert misinterprets questions from the knowledge engineer u consequences u incorrect knowledge, inappropriate solutions, solutions not found u remedy u formalized protocol for knowledge elicitation u validation of the knowledge base by domain experts

© Franz J. Kurfess Expert System Design 21 Syntax Errors u problem: rules or facts do not follow the syntax required by the tool used u knowledge engineer is not familiar with the method/tool u syntax not clearly specified u consequences u knowledge can’t be used u solutions u syntax checking and debugging tools in the ES development environment

© Franz J. Kurfess Expert System Design 22 Inference Engine Errors u problem: malfunctions in the inference component of the expert system u bugs u resource limitations v e.g. memory u consequences u system crash u incorrect solutions u existing solutions not found u remedy u validation and verification of the tools used

© Franz J. Kurfess Expert System Design 23 Inference Chain Errors u problem: although each individual inference step may be correct, the overall conclusion is incorrect or inappropriate u causes: errors listed above; inappropriate priorities of rules, interactions between rules, uncertainty, non- monotonicity u consequences u inappropriate conclusions u remedy u formal validation and verification u use of a different inference method

© Franz J. Kurfess Expert System Design 24 Limits of Ignorance Errors u problem: the expert system doesn’t know what it doesn’t know u human experts usually are aware of the limits of their expertise u consequences u inappropriate confidence in conclusions u incorrect conclusions u remedy u meta-reasoning methods that explore the limits of the knowledge available to the ES

© Franz J. Kurfess Expert System Design 25 Expert Systems and Software Engineering u software process models u waterfall u spiral u use of SE models for ES development u ES development models u evolutionary model u incremental model u spiral model

© Franz J. Kurfess Expert System Design 26 Generic Software Process Models u waterfall model u separate and distinct phases of specification and development u evolutionary development u specification and development are interleaved u formal systems development u a mathematical system model is formally transformed to an implementation u reuse-based development u the system is assembled from existing components [Sommerville 2001]

© Franz J. Kurfess Expert System Design 27 Waterfall Model [Sommerville 2001]

© Franz J. Kurfess Expert System Design 28 Suitability of Software Models for ES Development u the following worksheets help with the evaluation of software models for use in the development of expert systems u identify the key differences between conventional software development and ES development v with respect to a specific model u what are the positive and negative aspects of the model for ES development u evaluate the above issues, and give the model a score v 10 for perfectly suited, 0 for completely unsuitable u determine the overall suitability v high, medium low v explanation

© Franz J. Kurfess Expert System Design 29 AspectEvaluationScore key differences positive negative Waterfall Worksheet u overall suitability: high medium low u explanation

© Franz J. Kurfess Expert System Design 30 Evolutionary Development u exploratory development u objective is to work with customers and to evolve a final system from an initial outline specification. should start with well-understood requirements u throw-away prototyping u objective is to understand the system requirements. should start with poorly understood requirements [Sommerville 2001]

© Franz J. Kurfess Expert System Design 31 Evolutionary Development [Sommerville 2001]

© Franz J. Kurfess Expert System Design 32 AspectEvaluationScore key differences positive negative Evolutionary Dev. Worksheet u overall suitability: high medium low u explanation

© Franz J. Kurfess Expert System Design 33 [Sommerville 2001] Incremental Development u development and delivery is broken down into increments u each increment delivers part of the required functionality u user requirements are prioritised u the highest priority requirements are included in early increments u once the development of an increment is started, the requirements are frozen u requirements for later increments can continue to evolve

© Franz J. Kurfess Expert System Design 34 Incremental Development [Sommerville 2001]

© Franz J. Kurfess Expert System Design 35 Spiral Development u process is represented as a spiral rather than as a sequence of activities with backtracking u each loop in the spiral represents a phase in the process. u no fixed phases such as specification or design v loops in the spiral are chosen depending on what is required u risks are explicitly assessed and resolved throughout the process u similar to incremental development [Sommerville 2001]

© Franz J. Kurfess Expert System Design 36 Spiral Model Sectors u for quadrants in the coordinate system represent specific aspects u objective setting v specific objectives for the phase are identified u risk assessment and reduction v risks are assessed and activities put in place to reduce the key risks u development and validation v a development model for the system is chosen which can be any of the generic models u planning v the project is reviewed and the next phase of the spiral is planned [Sommerville 2001]

© Franz J. Kurfess Expert System Design 37 Spiral Model next phase Integration and test plan Development plan Requirements plan Life-cycle plan REVIEW [Sommerville 2001]

© Franz J. Kurfess Expert System Design 38 AspectEvaluationScore key differences positive negative Spiral Model Worksheet u overall suitability: high medium low u explanation

© Franz J. Kurfess Expert System Design 39 Formal systems development u based on the transformation of a mathematical specification through different representations to an executable program u transformations are ‘correctness-preserving’ u it is straightforward to show that the program conforms to its specification u embodied in the ‘cleanroom’ approach to software development [Sommerville 2001]

© Franz J. Kurfess Expert System Design 40 Formal Transformation Model [Sommerville 2001]

© Franz J. Kurfess Expert System Design 41 AspectEvaluationScore key differences positive negative Formal Transformations Worksheet u overall suitability: high medium low u explanation

© Franz J. Kurfess Expert System Design 42 Reuse-Oriented Development u based on systematic reuse u systems are integrated from existing components or COTS (commercial-off-the-shelf) systems u process stages u component analysis u requirements modification u system design with reuse u development and integration u this approach is becoming more important but still limited experience with it [Sommerville 2001]

© Franz J. Kurfess Expert System Design 43 Reuse-oriented development [Sommerville 2001]

© Franz J. Kurfess Expert System Design 44 AspectEvaluationScore key differences positive negative Reuse-Oriented Model Worksheet u overall suitability: high medium low u explanation

© Franz J. Kurfess Expert System Design 45 Generic System Design Process [Sommerville 2001]

© Franz J. Kurfess Expert System Design 46 System Evolution [Sommerville 2001]

© Franz J. Kurfess Expert System Design 47 Linear Model of ES Development u the life cycle repeats a sequence of stages u variation of the incremental model u once iteration of the sequence roughly corresponds to one circuit in the spiral model u stages u planning u knowledge definition u knowledge design u code & checkout u knowledge verification u system evaluation

© Franz J. Kurfess Expert System Design 48 Linear Model Diagram Planning Source Identification & Selection Acquisition Analysis & Extraction Definition Detailed Design Code & Checkout Formal Test Analysis System Evaluation Knowledge Definition Knowledge Design Knowledge Verification Knowledge Baseline Design Baseline Product Baseline Work Plan Knowledge Review Knowl. System Design Review Test Audit Review Final / Intermed. Review Prelim. Data Review Test Readiness Review

© Franz J. Kurfess Expert System Design 49 Planning u feasibility assessment u resource management u task phasing u schedules u high-level requirements u preliminary functional layout

© Franz J. Kurfess Expert System Design 50 Knowledge Definition u knowledge source identification and selection u source identification u source importance u source availability u source selection u knowledge acquisition, analysis and extraction u acquisition strategy u knowledge element identification u knowledge classification system u detailed functional layout u preliminary control flow u preliminary user’s manual u requirements specifications u knowledge baseline

© Franz J. Kurfess Expert System Design 51 Knowledge Design u knowledge definition u knowledge representation u detailed control structure u internal fact structure u preliminary user interface u initial test plan u detailed design u design structure u implementation strategy u detailed user interface u design specifications and report u detailed test plan

© Franz J. Kurfess Expert System Design 52 Code & Checkout u coding u tests u source listings u user manuals u installation and operations guide u system description document

© Franz J. Kurfess Expert System Design 53 Knowledge Verification u formal tests u test procedures u test reports u test analysis u results evaluation u recommendations

© Franz J. Kurfess Expert System Design 54 System Evaluation u results evaluation u summarized version of the activity from the previous stage u recommendations u as above u validation u system conforms to user requirements and user needs u interim or final report

© Franz J. Kurfess Expert System Design 55 Linear Model Exercise u apply the linear model to your team project u map activities, tasks, milestones and deliverables that you have identified to the respective stages in the linear model u use the linear model to sketch a rough timeline that involves two iterations v first prototype v final system u estimate the overhead needed for the application of the linear model in our context

© Franz J. Kurfess Expert System Design 56 Post-Test

© Franz J. Kurfess Expert System Design 57 Evaluation u Criteria

© Franz J. Kurfess Expert System Design 58 Important Concepts and Terms u evolutionary development u expert system (ES) u expert system shell u explanation u feasibility study u inference u inference mechanism u If-Then rules u incremental development u knowledge u knowledge acquisition u knowledge base u knowledge-based system u knowledge definition u knowledge design u knowledge representation u knowledge verification u limits of ignorance u linear model ES life cycle u maintenance u rapid prototyping u reasoning u rule u semantic error u software development life cycle u spiral development u syntactic error u waterfall model

© Franz J. Kurfess Expert System Design 59 Summary Expert System Design u the design and development of knowledge-based systems uses similar methods and techniques as software engineering u some modifications are necessary u the linear model of ES development is an adaptation of the incremental SE model u possible sources of errors are u knowledge and limits of knowledge errors u syntactical and semantical errors u inference engine and inference chain errors

© Franz J. Kurfess Expert System Design 60