1. Temporal Reasoning and Planning in Medicine Knowledge-Based Abstraction of Time-Oriented Clinical Data
Yuval Shahar, M.D., Ph.D

2. The Temporal-Abstraction Task
• Input: time-stamped data and relevant events
• Output: interval-based abstractions
• Identifies past and present trends and states
• Supports decisions based on temporal patterns
“modify therapy if the patient has a second episode of
Grade II bone-marrow toxicity lasting more than 3 weeks”
• Focuses on interpretation, rather than on forecasting

3. Temporal Abstraction: The Bone-Marrow Transplantation Domain

4. Uses of Temporal Abstractions
• Planning therapy and monitoring patients over time
• Creating high-level summaries of time-oriented patient records
• Supporting an explanation module for medical DSSs
• Representing goals and policies of therapy plans and guidelines
• Visualization and exploration of time-oriented medical data
last item:
Representing goals and policies in terms of maintaining
temporal patterns
thus enabling comparison of the system’s recommended
plan with that of the
human user, even if the immediate actions taken
seem to differ
e.g., both giving the patient blood and
attenuting the a toxic drug's dose might comply
with a higher-level pattern such as
"maintain Hb counts within 8-12;
avoid more than 2W of LOW Hb.")

5. Generation of Abstractions: The Knowledge-Based Temporal-Abstraction Method
Knowledge-based temporal abstraction (KBTA): A computational framework for interpretation of time-oriented data
Decomposes the TA task into five TA sub-tasks, each solved using one of five TA computational mechanisms, all mechanisms operating in parallel
Includes an explicit temporal-abstraction (TA) ontology (events, parameters, patterns, abstraction goals, contexts) and four TA knowledge types (structural, functional, logical, probabilistic)
Underlies tools for semi-automated acquisition of temporal-abstraction knowledge from domain experts

6. Knowledge-Based Temporal Abstraction

7. The Temporal-Abstraction Ontology (Shahar, AIJ, 1997)
• Events (interventions) (e.g., insulin therapy; bombardment)
- part-of, is-a relations
• Parameters (measured raw data and derived [abstract] concepts)
(e.g., hemoglobin values; anemia levels; number of open sockets)
- abstracted-into, is-a relations
• Patterns (e.g., crescendo angina; paradoxical hyperglycemia)
- component-of, is-a relations
• Abstraction goals (user views)(e.g., diabetes therapy; terror threats)
- is-a relations
• Interpretation contexts (effect of regular insulin; political tension)
- subcontext, is-a relations
• Interpretation contexts are induced by all other entities

8. Temporal-Abstraction Output Types
• State abstractions (LOW, HIGH)
• Gradient abstractions (INC, DEC)
• Rate Abstractions (SLOW, FAST)
• Pattern Abstractions (CRESCENDO)
- Linear patterns
- Periodic patterns
- Fuzzy patterns (partial match)

9. Temporal-Abstraction Knowledge Types
• Structural (e.g., part-of, is-a relations)
- mainly declarative/relational
• Classification (e.g., value ranges; patterns)
- mainly functional
• Temporal-semantic (e.g., “concatenable” property)
- mainly logical
• Temporal-dynamic (e.g., interpolation functions)
- mainly probabilistic

10. Dynamic Induction of Contexts: Temporal Constraints Between Inducing Proposition and Induced Context (Shahar, AMAI 1998) ee ss es se

11. Induction of Interpretation Contexts

12. The Meaning of Interpretation Contexts
Context intervals serve as a frame of reference for interpretation: Abstractions are meaningful only in a context (e.g., “Significant communication activity in the context of no mouse commands” )
Context intervals focus and limit the computations to only those relevant to a particular context (thus, knowledge is brought to bear only when relevant)
Contexts enable the use of context-specific knowledge, thus increasing accuracy of resultant abstractions

13. Advantages of Explicit Contexts
• Any temporal relation (e.g., overlaps) can hold between a
context and its inducing proposition; contexts can
be induced before and after the inducing proposition (thus
enabling a certain type of hindsight and foresight)
+ Claim: Forming contexts is a finite process
• The same context-forming proposition can induce multiple
context intervals
• The same interpretation context might be induced by
different propositions
• Explicit contexts support maintenance of several
concurrent views (or interpretations) of the data, in which
the same parameter has different values at the
same time, each within a different context
+ Note: There is no contradiction-values are in different contexts

14. Local and Global Persistence Functions: Exponential-Decay Local Belief Functions (Shahar, JETAI 1999) t j j
Bel(j) 1 2 I I 1 2 1
e –λ1t
e –λ2t j th
Time

15. Local and Global Persistence Functions and their Typology
Local (ρ) persistence functions represent the local persistence of the truth of a parameter proposition (a decay of the degree of belief forwards to the future, or backwards to the past), given a single parameter point or interval
Global (D) maximal-gap persistence functions return the maximal time gap that still allows us to join two propositions into an abstraction that is believed to be true, with a sufficient, task-specific, predefined degree of belief in the proposition, during the gap (a function of the parameter, its value, context, and the two interval durations)
An extension of local persistence functions that often can be constructed from them
Usually easier to acquire from domain experts since they can be linear functions of the durations
Global persistence functions can have four qualitative types: PP, NN, PN, and NP, depending on whether the D function is either (1) positive monotonic or (2) negative monotonic, with respect to (a) the length of the first parameter interval L(I1) or (b) the length of the second parameter interval L(I2)
Claim 1: PP D functions are associative (the order of joining intervals and points cannot change the resulting set of abstractions)
Claim 2: NN D functions are not associative
Can PN and NP D functions exist?

16. Irregular-Time Markov Models (Ramati & Shahar, UAI 2010)
Discrete-time Markov models do not handle time irregularity very well: Kalman Filters, Hidden Markov Models, and Dynamic Bayesian Networks in general require the specification of a constant time difference between each two consecutive observations
leads to inefficient computation or to information loss, and limits the inference to multiples of the modeled time granularity
Markov models that represent time continuously handle well time irregularity, but suffer from other limitations
they assume either a discrete state space (as Continuous-Time Bayesian Networks), or a at continuous state space (as stochastic differential equations)
Irregular-Time Bayesian Networks (ITBNs) generalize Dynamic Bayesian Networks such that each time slice may span over a time interval rather than a single time point, and the time difference between consecutive slices may vary according to the available data and inference needs
Leading to increased computational efficiency and increased expressivity

17. A Constraint-Based Specification of Linear and Periodic Patterns
(Chakravarty & Shahar, AMAI, 2000; MIM, 2001)
Periodic Pattern:
Periodic Constraints
Linear Patterns:
Global Constraints
<start>
<duration>
<end>
Abstractions:
<before>
Local Constraints
Raw Data:
Time

18. Linear and Periodic Clinical Patterns
Linear Component
Week 2
Week 3
Week 1
Anemia
Fever
Periodic Pattern
= Temperature
= Hemoglobin Level

19. An Overall View of the Temporal Abstraction Task
The temporal-abstraction task can be defined as follows:
Given at least one abstraction-goal interval, a set of event intervals , a set of parameter intervals, and the domain’s temporal-abstraction ontology, produce an interpretation - that is, a set of context intervals and a set of (new) abstractions - such that the interpretation can answer any temporal query about all of the abstractions derivable from the transitive closure of the input data and the domain knowledge.
This is, in fact, a knowledge-based data-integration task.

20. Application Domains for the KBTA Method [Shahar & Musen, Comp Biomed Res 1993, AI in Med 1996; Kuilboer et al., SCAMC 1993; Shahar & Molina, Patt Anal & App 1999; Boaz & Shahar, AI in Med 2005; Shabtai et al., J. Computer Virology, 2009; JIIS, 2012]
Medical domains:
Guideline-based care
AIDS therapy
Oncology [Shahar & Musen, CBR 1993, AIM 1996]
Monitoring of children’s growth [Kuilboer et al., SCAMC 1993]
Therapy of insulin-dependent diabetes [Shahar and Musen, AIM 1996]
Non-medical domains:
Evaluation of traffic-controllers actions [Shahar & Molina, PAA 1999]
summarization of meteorological data
Integration of intelligence data over time
Monitoring electronic security threats in computers and communication networks [Shabtai et al., JCV 2009; JIIS, 2012]

21. The RÉSUMÉ System Architecture

22. Monitoring of Children’s growth: The Parameter Ontology

23. Monitoring of Children’s growth: Temporal Abstraction of the Height Standard Deviation Score (HTSDS)

24. The Diabetes Parameter Ontology
= PROPERTY-OF relation;
= IS-A relation;
= ABSTRACTED_INTO relation

25. The Diabetes Event Ontology
= PART-OF relation;
= IS-A relation

26. The Diabetes Context Ontology
= SUB-CONTEXT relation;
= IS-A relation

27. Forming Contexts in Diabetes

28. Temporal Abstractions in Diabetes (I)T i m e ( d a t n 1 9 ) | B l o g u c s v M _ p 2 7 / 6 5 4 3 • H S r -
 = pre-breakfast glucose; • = pre-lunch glucose; D = pre-supper glucose

29. Temporal Abstractions in Diabetes (II)H T i m e ( d a t n 1 9 ) M | B l o g u c s v _ p 2 7 / 6 5 4 3 • L S r - .
 = pre-breakfast glucose; • = pre-lunch glucose; D = pre-supper glucose

30. Acquisition of Temporal-Abstraction Knowledge Using the Protégé System

31. Evaluation of Automated Knowledge Entry Using the Protégé System
Formal evaluation performed, using 3 experts, 3 knowledge engineers, 3 clinical domains, a gold standard of data, knowledge and output abstractions
Domains: monitoring of children’s growth, care of diabetes patients, and protocol-based care in oncology and AIDS.
The study evaluated the usability of the KA tool solely for entry of previously elicited knowledge.

32. KA Tool Evaluation: Results
Understanding RÉSUMÉ required 6 to 20 hours (median: 15 to 20 hours);
Learning to use the KA tool required 2 to 6 hours (median: 3 to 4 hours).
Acquisition times for physicians varied by domain: 2 to 20 hours for growth monitoring (median: 3 hours), 6 and 12 hours for diabetes care, and 5 to 60 hours for protocol-based care (median: 10 hours).
A speedup of up to 25 times (median: 3 times) was demonstrated for all participants when the KA process was repeated.
On their first attempt at using the tool to enter the knowledge, the knowledge engineers recorded entry times similar to those of the expert physicians’ second attempt at entering the same knowledge.
In all cases, RÉSUMÉ, using knowledge entered via the KA tool, generated abstractions that were almost identical to those generated using the same knowledge, when entered manually.

33. You can see the original guideline text on the right,
The GESHER Knowledge Structuring and Maintenance Tool: Creating a Declarative Knowledge Map from Medical Concepts
[Hatsek et al., OMIJ 2010]
A knowledge map
Constraints on concept values
Well, We start by building knowledge maps that define concepts such as toxemia of pregnancy, a disease whose main symptoms include high blood pressure;
In fact, we can zoom into an example and see how high BP is derived from high systolic BP or high diastolic BP;
You can see the original guideline text on the right,
the visual representation in the middle,
and the computer representation on the left
Structured text description

34. Detection of Infections in the ICU: The Knowledge
Then, we wondered if we can apply the same type of tools to the real time monitoring and diagnosis of actual patients; we started with detection of several types of infections in the Intensive Care Unit, or ICU.
You can see how we represented here the concept of problems in gas exchange, using a knowledge map, and how it corresponded to the text of the original guideline.
Centers for Disease Control (CDC) and Prevention. National Healthcare Safety Network. Guidelines and procedures for monitoring VAP. March

35. Blue: Additional data needed
The ICU Infection-Monitoring System
Red: Has an infection
Infection type
Patient ID
Blue: Additional data needed
Green: Normal
And this is how the interface of the actual ICU infection-monitoring system looks like;
For each infection type,
Green means everything is fine;
Red means an infection was detected;
Blue means a suspicion of an infection, but additional data not in the patient’s record are needed, such as X-ray images.
Displays the infections’ status of all the patients in the department in real-time
Can be queried and can provide explanations for all alerts

36. Evaluation of the GESHER Module [Hatsek et al., OMIJ 2010]
3 knowledge engineers, 3 physicians, 3 combined teams of KE+MD
All received a short training course in the Knowledge Map tool
All created the declarative knowledge specification of the Pre-eclampsia-Toxemia (PET) guideline (20 concepts)
Correctness and Completeness measures computed in comparison to a predefined expert+KE gold standard knowledge map

37. Results of the GESHER Evaluation
Group 1 vs. Group 2
Group 1 performance
Group 2 performance
P value
Physicians vs. Engineers
151/171 (88.3%)
162/171 (94.74%)
0.033 *
Physicians vs. Teams
165/171 (96.49%)
0.004 *
Engineers vs. Teams
0.428
Completeness
Group 1 performance
Group 2 performance
P value
Physicians vs. Engineers
603/663 (90.95%)
620/663 (93.51%)
0.081
Physicians vs. Teams
649/663 (97.89%)
<0.001 *
Engineers vs. Teams
<0.001*
Correctness
Training
Specification
Overall
Knowledge Engineers
100
188
288
Expert Physicians
140
313
453
Teams 93
193
287
Mean
111
231
342
Time
(minutes)
* significant

38. Conclusions
Temporal abstraction is an important task in medical decision- support systems, which can be solved using a knowledge-based methodology
The knowledge-based temporal-abstraction method employs reusable domain-independent temporal-abstraction mechanisms
The temporal-abstraction mechanisms rely on access to a domain-specific temporal-abstraction ontology of parameters, events, patterns, context, and abstraction goals
Temporal-abstraction knowledge can be specified either by knowledge engineers or by medical experts (best is by multidisciplinary teams) and is usable, reusable, and sharable