1. Theories in Formation (TIF)
DAY ONE: End-to End System and
CP Scenarios

2. Theories in Formation (TIF): Overview
Functionality:
Integrates knowledge acquired using separate interfaces and modalities (Cmaps, NL, sample cases, sketching, etc.)
Holds knowledge that is not yet in appropriate form for the KS and transforms it into formal representations
Relates the different knowledge inputs among themselves and to the existing KB and thus detects possible inconsistencies
Brings to bear relevant background k to help formalize inputs
Assists the user in resolving inconsistencies and other issues that may arise as these connections are being worked out
Contribution to the overall architecture:
Enable combined use of alternative input modalities
Proactive assistance to user entering new knowledge
Guard and/or inform K Server (KS) about possibly invalid statements

3. What can be Found in a Theory in Formation: 1) Content Knowledge
Formal and semi-formal knowledge structures
E.g., Cmaps, examples, analogies, sketches
Connections among k strucs (given or implied by user)
E.g., the “acid” mentioned in the virus degradation process is the same “acid” that appears in an earlier definition of a lysosome
Connections of new k to relevant background knowledge
E.g., the fermentors used in the Anthrax prod model are defined by the KS concept “fermentor-container”
Assertions derived or hypothesized by the system
E.g., “aflatoxin” is likely to be a virus based on what is said about it
New terms (not yet defined but known by reference)
E.g., the user mentioned the use of a “rotary evaporator” to dry toxins, but has not defined what it is (hyp 1: it is equipment; hyp 2: it is a stimulant substance)

4. What can be Found in a Theory in Formation: 2) Meta-knowledge
Alternative hypotheses and interpretations
E.g., user said Lybia produces Anthrax much like Irak does, yet user also stated that Lybia does not have chemfacs which implies it does not have centrifugal separators. Hyp 1: the dispersion process must be done differently than Irak’s; Hyp 2: there must be alt equpt other than cent. seps. that can be used
Source attributions
E.g., P was stated by user, Q was deduced by the KS, R was hypothesized by the TIF, S is the case in an example
Qualification statements and their rationale
E.g., assumption P generated correct answers in lines of reasoning R1…R7; S is inconsistent with T because...
Links to dialogue/session history

5. Theories in Formation FRINGE OF THE KS User’s input Text fragments
Established
connections
Hypotheses and
assumptions
Qualifications
Lines of reasoning
& other deductions
FRINGE OF
THE KS
Relevant background
knowledge
((( ))
())))
(defconcept
bridge ()))

6. Basic Principles to Focus Acquisition (I)
Principle of practical validation
Invalid/incomplete statements are more likely to appear in k fragments that have not been exercised by using them to solve problems or answer questions
Principle of experiential context
Invalid/incomplete statements are more likely to appear in k fragments where limited prior knowledge (theories, components, models, etc.) can be or has been brought to bear
Principle of local consistency
Inconsistencies are more likely to appear in k fragments that have not been defined and/or cannot be viewed in proximity (spatial, temporal, representational, or inferencial) by the user

7. Basic Principles to Focus Acquisition (and II)
Principle of purposeful knowledge capture
Additional knowledge should be sought until the system can perform the desired tasks, other additional knowledge is desirable but not strictly necessary
Principle of organized interaction (POI)
A system that is proactively assisting the user can be more effective if the interaction with the user is organized and set in context (by topic, by a typical acquisition strategy, by priority)

8. Basic Functions Knowledge formation Knowledge assimilation
Transform knowledge into a form adequate for the KS’s reasoners
Knowledge assimilation
Validate knowledge, note inconsistencies, detect gaps
Knowledge extraction
Guide acquisition of additional knowledge by generating and organizing follow-up questions to user

9. The Role of the KS KS (including analogy) is invoked to:
Generate deductions from certain sets of statements
E.g., given {…} what dual-use equipment does this country own
Solve problems or answer questions
E.g., PQs for EKCP, “views”
Suggest relevant models/theories/principles
E.g., could {…} fit the model of a “release process” and how?
Seek related cases or examples of certain statements
E.g., do you already know about any countries that seem to be capable of producing Anthrax but do not have fermentors?
Generate explanations
Perform KB diagnostics (KSL)

10. Structuring the User Interaction
Generating follow-up questions
Based on inconsistencies and knowledge gaps detected
Based on potential suggestions of applicable models/theories
Based on plausible hypotheses and assumptions generated
Organizing and prioritizing follow-up questions
Coherent dialogue: Easier for user if system brings up together questions on a topic
Adequate sequencing: The answers to some questions may help resolve others
KA strategies: guide user through typical KA tasks such as placing a new object within a hierarchy, filling attribute/value pairs through tables, specializing a process description, etc.

11. Integration with Overall Architecture
Knowledge Assimilation
Inconsistency detection
Establish connections/
lines of reasoning
K. server
API
Semi-structured
knowledge/
assertions
INTERACTION K
S E R V E R
MANAGER
Knowledge Formation
Newly
formalized
knowledge
Follow-up questions
Theories In
Formation (TIFs)
FRINGE OF
THE KB
Knowledge Extraction
Prioritize questions & structure interaction
Generate follow-up questions
Dialogue/
Session History

12. Development Plans Version 1 (Jan 2001?)
Capabilities:
Integrate inputs from alternative UIs (conc. maps, WebMod, WebNolgoy)
Push knowledge into KS, thus augmenting system’s Q/A capabilities
Generate follow-up acquisition questions to guide user to clarify/fix/resolve inconsistencies and other issues
Limitations:
Inputs from UIs are potential KS assertions (no semi-formal strucs)
All UIs acquire knowledge using the KS as guidance (e.g., all new processes need to be specified as extensions to existing process descriptions, no free-form input)
Version 2 (Jan 2002?): No limitations ;-)
Version 3 (??): Integrates inputs from multiple users
Techniques will be essentially the same
Will exploit Ocelot’s capabilities

13. Theories in Formation (TIF)
DAY TWO: Component Evaluations

14. Claims and Experiment Overview
Our claims are stated in terms of the basic principles used by the system
System will be developed so that each principle is operationalized in certain steps or submodules
Steps or submodules used to implement a principle can be disabled to build ablated versions of the tool
Gold standard: a consistent, complete body of domain knowledge that is removed from the KS for the experiment
Users re-enter that body of knowledge using full or ablated version of the system
Claims can be tested in a variety of contexts: at different stages of KA, with different kinds of k, etc.

15. Claim 1: Principle of Practical Validation
Principle of practical validation (PPV)
Invalid/incomplete statements are more likely to appear in k fragments that have not been exercised by using them to solve problems or answer questions
Claim:
Users will enter fewer incomplete/incorrect statements
because system follows PPV and thus
tests knowledge in the context of specific tasks

16. Claim 2: Principle of Experiential Context
Principle of experiential context (PEC)
Invalid/incomplete statements are more likely to appear in k fragments where limited prior knowledge (theories, components, models, etc.) can be or has been brought to bear
Claim:
Users will enter knowledge more efficiently & more correctly in
topics where system has relevant background k to bring to bear
because system follows PEC and thus
can use background knowledge to check, augment,
and structure new knowledge

17. Claim 3: Principle of Local Consistency
Principle of local consistency (PLC)
Inconsistencies are more likely to appear in k fragments that have not been defined and/or cannot be viewed in proximity (spatial, temporal, representational, or inferencial) by the user
Claim:
Users will enter fewer incorrect/invalid statements
because system follows PLC and thus
makes explicit the connections across knowledge
fragments and analyzes them

18. Claim 4: Principle of Purposeful Knowledge Capture
Principle of purposeful knowledge capture (PPKC)
Additional knowledge should be sought until the system can perform the desired tasks, other additional knowledge is desirable but not strictly necessary
Claim:
Users will enter knowledge more efficiently
because system follows PPKL and thus
focuses on acquiring knowledge relevant/needed
for intended tasks

19. Claim 5: Principle of Organized Interaction
Principle of organized interaction (POI)
A system that is proactively assisting the user can be more effective if the interaction with the user is organized and set in context (by topic, by a typical acquisition strategy, by priority)
Claim:
Users will enter knowledge more efficiently and more correctly
because system follows POI and thus
organizes clarification and correction questions
resulting in more effective communication of each
follow-up question and its context