1. Diagnostic Systems (I): Rule-Based Expert Systems and the MYCIN Project

2. Rule-Based Expert Systems: Suitable Domains
Many Rules
No Unifying Theorem
Knowledge can be easily separated from the way it is used
Updating the knowledge base has to be easy
The knowledge base can be the only [indirect] communication channel among rules
Clinical/psychological and other domains, rather than mathematical/physical domains

3. MYCIN: The Problem Roberts & Visconti [1972]:
Only 13% of patients are treated rationally
66% are being given irrational treatment
21% are being given questionable treatment
Irrationality means, for example:
Using a contra-indicated combination
Using the wrong agent for a specific organism
Not taking the required cultures

4. Stages in Diagnosis and Treatment
Decide if there is a significant infection
Identify the causing organism(s) by clinical and laboratory evidence
Decide what antibiotic agent the organisms are sensitive to
Prescribe the optimal drug combination for the particular case

5. A MYCIN Runtime Example

6. The MYCIN Architecture
Consultation program
Physician user
Static factual & judgmental knowledge
Dynamic patient data
Explanation program
Infectious diseases expert
Knowledge-acquisition program

7. The Knowledge Base Inferential knowledge stored in decision rules
If Premise then Action (Certainty Factor [CF])
If A&B then C (0.6)
The CF represents the inferential certainty
Static knowledge:
Natural language dictionary
Lists (e.g., Sterile Sites)
Tables (e.g., gram stain, morphology, aerobicity)
Dynamic knowledge stored in the context tree
Patient specific
Hierarchical structures: Patient, cultures, organisms
<Object, Attribute, Value> triples: <Org1, Identity, Strep>
A CF used for factual certainty <Org1, Identity, Staph, 0.6>

8. Example of a Decision Rule
IF:
The infection which requires therapy is meningitis
Organisms were not seen on the stain of the culture
The type of the infection is bacterial
The patient does not have a head injury defect
The age of the patient is between 15 and 55 years
Then:
The organisms that might be causing the infection are diplococcus-pneumoniae and neisseria-meningitidis

9. A Sample Context Tree

10. The Rule Interpreter Control structure: goal driven, backward chaining
Attempt to establish values of clinical parameters at the leaf nodes
The interpreter retrieves a list of rules whose conclusions bear on current goals, and tries to evaluate these rules
Questions are asked only when the rules fail to deduce the necessary information
If the user cannot supply the information, the rule is ignored

11. The Goal Rule RULE-092 IF: Then: Do the following: Else:
There is an organism which requires therapy
Consideration has been given to the possible existence of additional organisms requiring therapy, even though they have not actually been recovered from any current cultures
Then: Do the following:
Compile the list of possible therapies which, based upon sensitivity data, may be effective against the organisms requiring treatment
Determine the best therapy recommendations from the compiled list
Else:
Indicate that the patient does not require therapy

12. A MYCIN Reasoning Tree

13. The Main MYCIN Algorithm
Uses Monitor and FindOut to recursively invoke each rule when relevant

14. The Monitor Mechanism

15. The FindOut Mechanism

16. Certainty Factors
Not a Bayesian probability measure, but rather a Certainty Factor (CF) with its update functions
A Conclude function uses
The CF of the rule used for making the inference
The minimal CF of the premises (using the Tally function)
The context node about which the conclusion is made
The clinical parameter whose value is added to the dynamic DB
The value of the clinical parameter
Conclude derives a conclusion including the CF of the result
E.g., “There is suggestive evidence (0.7) that the identity of the organism is streptococcus”
The CF is mapped into English
The CF of a context is updated by other evidence (relevant rules)
It is always true that -1 ≤ CF ≤ +1
If CF = +1 then all other hypotheses are rejected

17. Question Answering General question answering module
Used after a consultation regarding any aspect of it
General questions about the knowledge base (e.g., “is blood a sterile site?”)
Uses a simple keyword pattern-matching approach
Online reasoning status checker provided during consultation
A limited set of explanations for the reasoning line
HOW: recapitulates all relevant successful rules used on the path to the conclusion asked about
WHY: backtracks to the reason that a question (a particular bit of information) was asked

18. Example: The WHY Explanation Capability
In this series, how many of the cultures were positive for the e.coli {typical system prompt}
(ORGANISM-1)?
—WHY {request from user to determine motivation for question}
[i.e. WHY is it important to determine the number of cultures which {system’s rephrasing of the WHY question, were positive for the organism?] and answer}
[1.0] This will help to determine whether there is significant disease
associated with this occurrence of the organism.
It has already been established that
[1.1] the culture was taken from a sterile source
[1.2] the number of cultures in the series is greater than 2
Therefore, if
[1.3] the number of cultures which were positive for ORGANISM-1
is greater than 2
Then there is suggestive evidence (.5) that there is significant disease
associated with this occurrence of the e.coli (ORGANISM-1).
[RULE049]
— WHY {repeated WHY to determine relevance of previous explanation}
[i.e. WHY is it important to determine if there is significant disease associated with this occurrence of the organism?]
[2.0] This will help to determine a diagnosis for which the infection should be treated.
it has already been established that
[2.1] the infection is not one of: primary-bacteremia meningitis
[2.21 for some current culture of the patient it is true that the site of the culture is blood
[2.3] there is significant disease associated with this occurrence of the organism
Then
it is definite (1.0) that the diagnosis for which the infection should be treated is secondary-bacteremia
[RULE103]

19. Knowledge Acquisition
The knowledge base (KB) has to be modified and expanded continuously
The knowledge-acquisition (KA) bottleneck
The TEIRESIAS module enabled interactive KA from medical experts
Included a rule model for different types of rules, which creates expectations about the structure of the acquired rule
A new or old rule can be created or modified interactively using meta-knowledge about rule models
KA important also in the context of a reasoning error

20. Therapy Selection
Originally a combination of Therapy Rules and a LISP procedure
A list of potential therapies is created
The best combination of drugs is selected
Resulted in a context tree of possible therapies under the “current organism” node
Replaced by a clearer version enabling explicit explanations, as well as optimal therapy, using a Plan, Generate, and Test strategy
more appropriate, since the therapy-planning task is really a configuration task

21. Improved Therapy Selection
Plan by re-ranking the potential drugs, using local factors (e.g., organism sensitivity, drug toxicity, current therapy continuity)
Propose a recommendation and Test it, using global factors (e.g., minimize total number of drugs)
Proposals are managed using a canonical instruction set
Testing uses rules to check for proper coverage, unique drug classes, and patient-specific recommendations
The first satisfactory proposal is chosen
Prescribe final recommendation
Uses algorithmic dosage calculation and patient-specific adjustment

22. Clinical Evaluation of MYCIN [Yu et al., Comp. Prog. Biomed. 9, 1979]
Objective assessment of the basic (bacteremia) system
Examined the main three decision-making steps:
Decide if there is a significant organism
Determine organism identity
Recommend therapy, including alternatives

23. The Evaluation Method
15 patients with positive blood cultures (at least one organism)
5 Stanford infectious disease experts
5 experts from other hospitals
All data recorded and given, if asked for, by the computer or a human expert
All decisions by the computer or the experts recorded, including the majority opinion

24. Results of the Evaluation Study
Significant-organism decision:
MYCIN decided identically in 97% of the 150 (15x10) decisions
MYCIN decided identically in 1000% of the 15 majority decisions
Organism-identification decision:
MYCIN identified 45 organisms in the 11 cases requiring therapy, thus 450 (45x10) organism judgements
Agreement in 80% of Stanford experts’ judgements, 72% of others
Agreement by the majority of experts: 90%
Treatment decision: treatment was suggested to all 11 patients requiring it, thus 110 (11x10) decision instances to compare
Agreement with MYCIN’s recommendation was 76% for Stanford experts, 69% for the others
Agreement by a majority: 90% for Stanford experts, 73% for the others
In one case, agreement of 4/5 Stanford experts, disagreement of 4/5 others!
the KB represented well the Stanford prescription habits
Overall “acceptable” performance was rated in 93% (14/15) of cases
Several design problems, such as unblinded evaluation of the program’s and experts’ performance, were corrected in a later study
MYCIN was actually ranked best, relative to all other experts, in a blinded evaluation

25. Summary: Rule-Based Expert Systems and the MYCIN Project
Task: Diagnosis and treatment of infectious diseases
Problem solving method: Heuristic Classification [Clancey, 1985]
Data->Abstracted data=>Abstracted solutions->Solutions
Implementation: Backward-chaining production rules
Evaluation results: Surprisingly good for a research tool
Different evaluation by Stanford and Other experts stems probably from different local practices. This might be actually considered as a representational success. (Rules and tables can be modified easily).
Many technical and conceptual problems prevented clinical use (small memory, slow CPU, medical DB communication problems, stand-alone system, etc.), several of which are now solvable
At the time of the first study, MYCIN rules included only bacteremia (meningitis and endocarditis were added later), thus never tested in a real clinical environment with general infections
Practically no temporal reasoning
Implicit control — hard to modify
Probabilistic model was not Bayesian and not intuitive
The knowledge-acquisition bottleneck remained significant