1. Hybrid Systems Two examples of the Combination of Rule-Based Systems and neural nets By Pieter Buzing

2. Plan 4 Introduction: –Knowledge Based System vs Neural Net –Basic hybrid technique 4 Fu’s system 4 KBANN system 4 Comparison (rule improvement, semantics) 4 Conclusions

3. Introduction 4 Knowledge Based System 4 Neural Network 4 Characteristics KBS & NN 4 Basic hybrid technique

4. Knowledge Based System 4 Rule base and fact base 4 Facts   Conclusions 4 Certainty Factors [-1, 1] 4 IF smart AND ambitious rich 4 Given: CF(smart)=0.8 CF(ambitious)=0.5 4 Conclude: CF(rich)=0.7*min(0.8, 0.5)=0.35 INFERENCE CF=0.7

5. Neural Network 4 Nodes and connections 4 layers: input, hidden and output nodes 4 Aim: right weight vector for each connection 4 Trained with examples: minimize error

6. Characteristics

7. Basic hybrid technique Initialize the neural network with domain knowledge. So architecture and initial weight are now founded! Use the following mapping:

8. Fu’s system (1989) 4 Proposed by: Li-Min Fu, Winsconsin 4 Objective: let NN deal with incorrect KB 4 Construction: conceptual network with CFs AND-nodes to maintain meaning 4 Training: backpropagation and hill-climbing because AND-function not differentiable 4 Error handling: identifies wrong rules 4 Semantics: rules always ‘visible’ in network

9. KBANN system (1994) 4 Proposed by: Towell&Shavlik, Winsconsin 4 Objective: use KB to initialize a NN 4 Construction: concept  node extra nodes and connections added 4 Training: backpropagation 4 Error handling: weight adjustment 4 Semantics: too many connections to make sense out of it

10. Comparison (1) Coping with erroneous rules 4 Fu considers rule incorrect when weight change exceeds a threshold 4 KBANN deals with it implicitly, it alters the weight of a inconsistent rule 4 Fu can identify malicious rules when 12% of rules is corrupted 4 KBANN: outperforms standard NN with 10% big or 30% small changes

11. Comparison (2) Maintainability of semantics 4 Fu: every unit keeps its meaning 4 KBANN: (random) units are added 4 Fu: conjunction units hold their original semantic basis 4 KBANN: all nodes are connected, so every node is a big ‘conjunction’ 4 Fu’s weights are CFs. KBANN?

12. Conclusions 4 Coping with erroneous rules –Fu can be used to verify rules: identify inconsistent ones. –KBANN handles it convincingly 4 Maintainability of semantics –Fu succeeds in comprehensibility goal –KBANN loses its semantics: mere starting base 4 Mind you: different goals, periods, domain