1. Unclassified//For Official Use Only 1 RAPID: Representation and Analysis of Probabilistic Intelligence Data Carnegie Mellon University PI : Prof. Jaime G. Carbonell / jgc@cs.cmu.edu / (412) 268-7279 Dr. Eugene Fink / e.fink@cs.cmu.edu / (412) 268-6593 Dr. Anatole Gershman / anatoleg@cs.cmu.edu / (412) 268-8259 DYNAMiX Technologies POC: Dr. Ganesh Mani / gmani@dynamixtechnologies.com / (412) 401-0121 Mr. Dwight Dietrich / ddietrich@dynamixtechnologies.com / (724) 940-4304 PAINT

2. Unclassified//For Official Use Only 2 RAPID: Product RAPID is a software system for the analysis of dynamically evolving intelligence, which will help analysts with the following tasks. Draw probabilistic conclusions from available intelligence, including uncertain and missing data Identify potentially surprising developments Answer questions Formulate and assess hypotheses Identify critical uncertainties Develop strategies for proactive collection of additional data to resolve critical uncertainties

3. Unclassified//For Official Use Only 3 RAPID: Product Initial knowledgeIntelligence results Available knowledge Observable facts Hidden facts Knowledge sources: Public domain Intelligence Inferences The system will help: Identify important holes Locate most crucial missing pieces Insert them into the puzzle Jigsaw analogy:

4. Unclassified//For Official Use Only 4 RAPID: Product Knowledge base Adversarial goals Inference rules Task hierarchy Uncertain situation assessment Massive databases, including both certain and uncertain data Indexing and retrieval Fast search for exact and approximate matches Automated construction of new rules and Markov models Learning of new knowledge Massive new intelligence PROACTIVE INTELLIGENCE CONTROL Identification of critical uncertainties Contingency analysis Adversarial search Tools for manual modification of the available knowledge Knowledge editing Analyst GUI Hypotheses, conclusions, and data-collection plans Markov models Explanation of inferences Functionality: Representation of uncertainty Inferences from uncertain data Analysis of critical uncertainties Proactive intelligence plans Predictive Markov models Graphical user interface General intelligence collection Proactive intelligence collection Hypotheses, conclusions, and data-collection plans Massive new intelligence

5. Unclassified//For Official Use Only 5 RAPID: Availability Version 1: Scalable probabilistic matching of uncertain intelligence data July 2008 Version 2: Discrimination among competing hypotheses based on fast probabilistic inference; proactive identification of critical uncertainties July 2009 Version 3: Advanced proactive-intelligence planning; learning of hypotheses and inference rules; graphical user interface July 2010 Version 4: Analysis of contingencies and adversarial goals; explanation of inferences July 2011 Version 5: Full-featured deliverable systemDec. 2011 All versions of RAPID will demonstrate all capabilities, with increasing functionality over time, with primary emphasis on:

6. Unclassified//For Official Use Only 6 RAPID: Product System requirements: High-end Windows desktop computer OS: Windows XP or Vista Software: Visual Studio.Net, Java Memory: 4 GByte CPU: 3 GHz Hard drives: 1 TByte Distribution model: Executable code and documentation

7. Unclassified//For Official Use Only 7 RAPID: Technology Proactive collection of intelligence: Automated identification of critical uncertainties, based on given tasks and priorities Planning of proactive intelligence collection, based on the analysis of cost/benefit trade-offs and related risks Contingency analysis of alternative scenarios related to the plans for proactive intelligence collection and prevention of possible adversarial actions Filtering and processing of new intelligence Propagation of inferences Analysis of key indicators Development of intelligence- collection plans Analyst Massive new intelligence Intelligence collection

8. Unclassified//For Official Use Only 8 Underlying technology: Management and analysis of massive amounts of structured uncertain data, including intelligence reports, analyst opinions, inference rules, hypotheses, and data-collection plans. RAPID: Technology Novel representation of massive amounts of uncertain data, which supports fast matching and inferences Scalable inference mechanism for reasoning about uncertain intelligence Application of predictive Markov models to analyze alternative hypotheses and future developments Construction of optimized intelligence-collection plans Integrated graphical user interface for collaboration between the system and human analysts

9. Unclassified//For Official Use Only 9 RAPID: Technology Novel representation of uncertain data: Uncertain nominals, numbers, strings, spatial data, graph topologies, and functions Indexing of massive uncertain data, and fast retrieval of exact and approximate matches Control of the trade-off between the expressiveness and reasoning speed Possible values 250 5007501000 Probability Example: A relatively reliable source indicates that the number of members in a terrorist group is between 250 and 500 An alternative, somewhat less reliable source indicates that this number may be between 750 and 1000 0.75 0.25

10. Unclassified//For Official Use Only 10 RAPID: Technology Novel inference mechanism: Representation of dependencies among data by “if-then” and numeric inference rules Fast propagation of inferences through large- scale probabilistic networks of dependencies, based on massive uncertain data Terrorist group size Available funds Ties with other groups Possibility of chemical attack If-then inference Numeric inference

11. Unclassified//For Official Use Only 11 Development Goal RAPID: Technology Application of predictive Markov models: Evaluation of given hypotheses Identification of key indicators Analysis of possible future developments Automated improvement of inferences, hypotheses, and predictive models WMD facilities Material acquisition Qualified personnel Available material Facility construction Personnel hiring X1X1 Z1 1 Z2 1 Y1Y1 Obser- vations Hidden reality New obser- vations Develop- ment goal X2X2 Z1 2 Z2 2 Y2Y2 Past Present

12. Unclassified//For Official Use Only 12 RAPID: Technology Graphical user interface: Integrated access to all system components Visualization and explanation of intelligence data, hypotheses, and proactive intelligence-collection plans Collaboration between the system and human analysts

13. Unclassified//For Official Use Only 13 RAPID: Technology Innovative claims: Representation of massive uncertain data that supports fast matching and inferences Automated analysis of critical uncertainties and development of intelligence-collection plans Automated identification of hidden patterns in massive amounts of uncertain intelligence data

14. Unclassified//For Official Use Only 14 RAPID: Evaluation We expect that the RAPID system will provide significant advantage over available off-the-shelf tools, such as standard database systems. To support this claim, we expect to compare the productivity of analysts using RAPID with that of analysts who perform the same tasks using commercially available tools. Experimental group: Use of RAPID Control group: Use of standard tools

15. Unclassified//For Official Use Only 15 RAPID: Evaluation Experimental setup: We expect to recruit retired intelligence analysts for the system evaluation, and ask them to perform several tasks based on given uncertain data. Identify the data most relevant to given tasks Evaluate the validity of given hypotheses Find relevant hidden patterns Identify critical missing data, and propose a cost-effective plan for collecting this data

16. Unclassified//For Official Use Only 16 RAPID: Evaluation Performance measurements: We will measure the following main factors to evaluate the performance of analysts: Number of high-level tasks completed within the experiment time frame Accuracy of hypothesis evaluation Number and relevance of identified patterns Effectiveness and costs of proactive data-collection plans We will also ask analysts to complete a questionnaire on their overall experience with the system.

17. Unclassified//For Official Use Only 17 RAPID: Evaluation Expected results: We will view the proposed work as success if RAPID consistently outperforms the off-the- shelf tools in all four performance factors, the performance difference for each factor is statistically significant, and analysts report the overall positive experience of using the system.

18. Unclassified//For Official Use Only 18 RAPID: Contributions Feedback Data Strategy Generation and Exploration Dynamic Simulation Models Response Options 2 3 4 Representation of massive uncertain knowledge Automated discovery of casual relationships User-guided learning of predictive models Fast probabilistic integration of all evidence Analysis of possible future developments 1 Identification of critical uncertainties and surprises Development of proactive cost/benefit weighted intelligence-gathering plans 1 4 3

19. Unclassified//For Official Use Only 19 RAPID: Contributions Inputs and outputs: Uncertain intelligence and analyst opinions: Massive stream of structured records Specific hypotheses New learned knowledge Data-search queries Query matches Evaluation of hypotheses Plans for proactive collection of intelligence data Uncertain situation assessment Domain knowledge: Inference rules and Markov models Domain knowledge RAPID General intelligence collection Proactive intelligence collection

20. Unclassified//For Official Use Only 20 RAPID: Contributions Inputs: From other PAINT components: Available intelligence data and its certainty Hypotheses about unknown factors and their certainty Available domain knowledge From analysts: Intelligence-analysis tasks and priorities Hypotheses and related opinions Responses to RAPID-generated probes Additional domain knowledge From other sources: Databases with available intelligence Public databases with relevant data

21. Unclassified//For Official Use Only 21 RAPID: Contributions Outputs: Inferences from available uncertain data Evaluation of given hypotheses New hypotheses about unknown factors and their certainties Cost/benefit weighted plans for proactive intelligence gathering Learned inference rules and Markov models

22. Unclassified//For Official Use Only 22 RAPID: Domain Strategic technology domain: Tracking the developments in strategic technologies, and the related capabilities of countries and organizations. Known and emerging technologies with potential military applications Technology experts, their affiliations, professional networks, speeches, conference attendance, etc. Organizations interested in these technologies, their affiliations, ownership, alliances, supply and distribution networks, and potential intentions Pronouncements by business and political leaders regarding strategic technologies Financial information related to these technologies Example: Iran’s plans with respect to nanotechnology.

23. Unclassified//For Official Use Only 23 RAPID: Backup domain Network security domain: Tracking computer-network traffic and identifying potential software and hardware problems, as well as malicious network activity. Normal network patterns and their changes over time Types and symptoms of “natural” network problems, such as overloading and hardware failures Types of unusual network activity, and signs of malicious activity The work on this domain will be secondary to the work on the strategic-technology domain.