Verification and Validation To ensure quality in expert system verificaion and validation are important steps in the development lifecycle. Based on Chap. 16: The Engineering of Knowledge-based Systems: Theory and Practice, A. J. Gonzalez and D. D. Dankel

Not Complying with the system specs. Semantic and syntactic errors Errors may stem from: Not Complying with the system specs. Semantic and syntactic errors Incorrect representation of the knowledge domain

Verification and validation in expert systems is different from that of conventional expert system ES are not purely objective Uncertainty is accepted and may even be encouraged ES output cannot be easily verified through experiments Expert (or group of experts) decide on the accuracy of the system

Verification The adherence to the system specification and correct implementation process.

Compliance with specs Knowledge representation Reasoning technique Modularity Interfaces Explanation facility Performance Maintainability Security

Redundant rules If a is true and b is true then the temp is -40 0 C Identical rules or rules that have the same meaning: If a is true and b is true then the temp is -40 0 C then the temp is -40 0 F

Conflicting rules If a is true and b is true then x then ~x Identical premises with conflicting conclusions: If a is true and b is true then x then ~x

Subsumed rules If a is true and b is true then x All the premises of one rule is a subset of another rule, and both have identical conclusion: If a is true and b is true then x If a is true and b is true and c is true

Circular rules If a is true and b is true then x and y Conclusion of rule1 leads to rule2 firing, whose conclusion leads to rul1 firing: If a is true and b is true then x and y If x is true and y is true then a and b

Unnecessary IF conditions Two rules have identical conclusion, but one of the premises is conflicting: If a is true and b is true and ~c is true then x If a is true and b is true and c is true

Dead-end rules A rule whose action is not one of the conclusions of the system or whose conclusion is not used in the premise of any other rule.

Missing rules Dead end rules could be caused by the removal of rules or the failure of the knowledge engineer to include a rule.

Unreachable rules The premise of the rule will never ever be true.

Validation Checks to see if the system is correct, i.e. the output is consistent with input.

Validation issues What is being validated? Method Criteria When should validation occur?