ExaSphere Network Analysis Engine © 2006 Joseph E. Johnson, PhD www.exasphere.comwww.exasphere.com.

Slides:



Advertisements
Similar presentations
2009 Transition Dynamics Enterprises, Inc. Used with permission. What You Need to Know about the Green Economy Add your name here.
Advertisements

SensMax People Counting Solutions Visitors counting makes the most efficient use of resources - people, time and money, which leads to higher profits in.
BITTOR A New Foundation for Logical & Numeric Uncertainty Technical Overview Markov Monoids are used as a Mathematical Foundation for a New Theory of Logical.
Modeling and simulation of systems Slovak University of Technology Faculty of Material Science and Technology in Trnava.
DARPA Oasis PI Meeting Hilton Head, SC March 2002.
NETWORK ANALYSIS BASED UPON THE RENYI' ENTROPIES OF THE ASSOCIATED MARKOV MONOID TRANSFORMATION. USC CAS IMI Summer School on Network Science May 2013.
Computational Methods for Management and Economics Carla Gomes Module 9a Network Models Terminology (Slides adapted from J.Orlin’s)
Networks, Lie Monoids, & Generalized Entropy Metrics Networks, Lie Monoids, & Generalized Entropy Metrics St. Petersburg Russia September 25, 2005 Joseph.
Introduction and Overview “the grid” – a proposed distributed computing infrastructure for advanced science and engineering. Purpose: grid concept is motivated.
1 SYS366 Week 1 - Lecture 2 How Businesses Work. 2 Today How Businesses Work What is a System Types of Systems The Role of the Systems Analyst The Programmer/Analyst.
Definition Reliability is a general quality of an object – an ability to perform a desired function, sustaining the values of rated operational indicators.
National Security Brittany Haga Sean Bevans Kelsey Splinter.
Multi-Scale Analysis for Network Traffic Prediction and Anomaly Detection Ling Huang Joint work with Anthony Joseph and Nina Taft January, 2005.
1 Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information? TS Eliot, 1934.
Transportation Models Transportation problem is about distribution of goods and services from several supply locations to several demand locations. Transportation.
1 Network Metrics: A Network Analysis Engine ICCS 2006 Conference June 2006 Boston MA Joseph E. Johnson, PhD Department of Physics University of South.
Soft. Eng. II, Spr. 2002Dr Driss Kettani, from I. Sommerville1 CSC-3325: Chapter 9 Title : Reliability Reading: I. Sommerville, Chap. 16, 17 and 18.
Geographic Information System - ArcView University at Buffalo Summer Institute 2003 May 12, 2003.
Conceptual Modeling of the Healthcare Ecosystem Eng. Andrei Vasilateanu.
CT2 Strategy Foundation Who We Serve: Mid Cap to Large Companies who consume multi-mode shipping services. Our Purpose: To achieve the status as ‘market.
Location Models For Airline Hubs Behaving as M/D/C Queues By: Shuxing Cheng Yi-Chieh Han Emile White.
An Information Theory based Modeling of DSMLs Zekai Demirezen 1, Barrett Bryant 1, Murat M. Tanik 2 1 Department of Computer and Information Sciences,
Achieving Better Reliability With Software Reliability Engineering Russel D’Souza Russel D’Souza.
Chapter 1 Introduction to Simulation
Networks and the Shortest Path Problem.  Physical Networks  Road Networks  Railway Networks  Airline traffic Networks  Electrical networks, e.g.,
Database Design – Lecture 16
Performance analysis and prediction of physically mobile systems Point view: Computational devices including Mobile phones are expanding. Different infrastructure.
Tutor: Prof. A. Taleb-Bendiab Contact: Telephone: +44 (0) CMPDLLM002 Research Methods Lecture 8: Quantitative.
1. There are different assistant software tools and methods that help in managing the network in different things such as: 1. Special management programs.
Electrical and Computer Systems Engineering Postgraduate Student Research Forum 2001 WAVELET ANALYSIS FOR CONDITION MONITORING OF CIRCUIT BREAKERS Author:
1 The Research on Analyzing Time- Series Data and Anomaly Detection in Internet Flow Yoshiaki HARADA Graduate School of Information Science and Electrical.
CS532 Term Paper Applying Six Sigma Methodology In Software Engineering Sourabh Bandyopadhyay.
ECONOMIC RESOURCES Also called: Factors of Production and Economic Inputs.
A performance evaluation approach openModeller: A Framework for species distribution Modelling.
Design for Engineering Ten Major Branches of Engineering Technology Education 660 Unit 1 14 April, Greg Heitkamp This material is based upon.
University of Southern California Center for Systems and Software Engineering Model-Based Software Engineering Supannika Koolmanojwong Spring 2013.
Database Design and Management CPTG /23/2015Chapter 12 of 38 Functions of a Database Store data Store data School: student records, class schedules,
Client: The Boeing Company Contact: Mr. Nick Multari Adviser: Dr. Thomas Daniels Group 6 Steven BromleyJacob Gionet Jon McKeeBrandon Reher.
J1879 Robustness Validation Hand Book A Joint SAE, ZVEI, JSAE, AEC Automotive Electronics Robustness Validation Plan The current qualification and verification.
Introduction Infrastructure for pervasive computing has many challenges: 1)pervasive computing is a large aspect which includes hardware side (mobile phones,portable.
Business Data Communications, Fourth Edition Chapter 11: Network Management.
Supernetworks: Paradoxes, Challenges, and New Opportunities Anna Nagurney John F. Smith Memorial Professor The Virtual Center for Supernetworks.
ECONOMIC RESOURCES Also called: Factors of Production and Economic Inputs.
Alexey A. Didyk Kherson national technical university, Ukraine
Introduction to IT Lecture – 01.
CHAPTER 8 INTERORGANIZATIONAL AND GLOBAL INFORMATION SYSTEMS 1/9.
Performance Validation of Mobile IP Wireless Networks Presented by Syed Shahzad Ali Advisor Dr. Ravi Pendse.
FM - AGS 2001 What computers are good at ? The advantages of ICT.
Video sX5E&index=6&list=PLJ8uEbBRJZKf878RK5gjm DvSciU4AUNmY&spfreload=10
MANAGEMENT INFORMATION SYSTEM
11-1  Material requirements planning (MRP): Computer-based information system that translates master schedule requirements for end items into time-phased.
Fundamentals of Information Systems Dr. Hanan Moussa.
Chapter 11 Information Systems Within the Organization.
Cyber security: Lithuanian National Regulatory Authority expertise in monitoring national networks resilience Dr. Rytis Rainys | rrt.lt at TAIEX Multi-beneficiary.
Zero Day Attacks Jason Kephart. Purpose The purpose of this presentation is to describe Zero-Day attacks, stress the danger they pose for computer security.
::Network Optimization:: Minimum Spanning Trees and Clustering Taufik Djatna, Dr.Eng. 1.
Student Name USN NO Guide Name H.O.D Name Name Of The College & Dept.
Introduction To Modeling and Simulation 1. A simulation: A simulation is the imitation of the operation of real-world process or system over time. A Representation.
(1) Organize information processing centers environment, the various functions and details Electronic Data Processing (EDP): can refer to the use of automated.
A Multi-Airport Dynamic Network Flow Model with Capacity Uncertainty
Lesson 4: Computer APPLICATIONS
Discovering Computers 2010: Living in a Digital World Chapter 14
QianZhu, Liang Chen and Gagan Agrawal
Empirical analysis of Chinese airport network as a complex weighted network Methodology Section Presented by Di Li.
Blockchain Adrian Zaragoza.
A Real-time Intrusion Detection System for UNIX
Also called: Factors of Production and Economic Inputs
Chapter-5 Traffic Engineering.
Overview of Intermodal (Multimodal) Supply Chain Optimization and Logistics Scott J. Mason, Ph.D. Fluor Endowed Chair in Supply Chain Optimization and.
Yining ZHAO Computer Network Information Center,
Presentation transcript:

ExaSphere Network Analysis Engine © 2006 Joseph E. Johnson, PhD

The Network Problem  Networks are vast and complex contain astronomical amounts of data  Yet there has been no method of tracking their behavior. Until NOW

What is a Network?  A network is a set of points (or nodes) with some of the points connected.  Examples: Airline networks – airports connected by the number of flights or passengers. Internet networks – computers connected by the amount of information sent. Financial networks – bank accounts connected by the amount of fund transfer.

ExaSphere: Origin & Purpose  The ExaSphere Network Engine is the product of years of research funded at over $2.5M by DARPA (US Dept. of Defense).  The ExaSphere Network Engine is a general purpose software solution that can be applied to any network for tracking and analysis.

Why is the Problem Difficult?  Networks contain millions of nodes and trillions of changing values.  Yet all values are equally important. Thus the quantity of data is overwhelming.  Networks are among the most difficult of generally unsolved mathematical problems.  We need to reduce this vast data to a few representative values – called network metrics.

Where can this software be applied?  Communications Networks Internet Traffic: malicious process detection Phone (land & wireless): system behavior Mail (UPS, FedEx..): flow monitoring  Transportation: patterns & demand Air Traffic Roadways (personal & trucking) Railroads Pipelines International Shipping

Also:  Financial Banking Transactions Monitoring Accounting Flows Ownership & Investments  Utility & Energy Electrical Grids Water & Sewer Flows Natural Gas & Petroleum Distribution

And Also:  Social Networks Organizational & Business Criminal & Terrorist Networks  Manufacturing Process Supply Chain Manufacturing Workflow  Health & Biological Disease Ecological

How Does it Work - 1  The data input has four parts in each record: (t, i, j, w) 1. Date-Time of the action (t) 2. The originating node identifier (i) 3. The receiving node identifier (j) 4. The weight or value of the connection (w)  These weights are added into a matrix, C ij over an interval of time (minute, day,…)

How Does it Work - 2  New mathematical discoveries are invoked:  The ‘connection matrix’ C ij is processed to a set of values that represent the order and disorder (entropy metrics) of the connectivity structure for each node.  These metrics are tracked as a curve in real time that reveals abnormal changes and at which nodes they occur.

Natural Time Frames:  Internet Traffic and Electrical Grids must be rapidly monitored perhaps every minute.  Banking Transactions have a natural time of perhaps a day as would most Transportation Flows while Accounting Flows have a more natural time of one month.  Criminal Networks would probably build a more permanent connection structure.

Expected Changes  Certain structural changes are natural to different networks and such behavior has to be considered to identify anomalies: Internet, telephone, and transportation flows depend upon the time of day and day of the week. Electrical grids are highly dependent upon the weather as well as the time of day.

The ExaSphere Network Analysis Engine  - is the first and only completely generalized tool for network monitoring  - tracks the entropy spectra over time identifying abnormal behaviors, attacks, and system failures, by node  - is a general purpose software engine solution for all types of networks

Software Environment  The core software components have been built in both professional JAVA and Mathematica languages independently.  Current work: Expand the user tools. Develop success stories for different network applications. Continue our analysis of internet behavior at two sites.

End of Business Overview  Detailed information is available at

Mathematical Overview -1  The inventor discovered that every possible network structure, C ij, corresponds to exactly one infinitesimal generator of a Markov transformation.  Each such Markov transformation consists of columns that are valid probability distributions (non-negative values normed to unity).

Mathematical Overview - 2  Each such column vector represents a transformation probability profile toward a node.  Shannon and generalized Renyi entropy functions can then be rapidly computed on each column giving the order / disorder of the probability profile for that node.  These entropy metrics can be sorted in order to represent the entropy spectra (a curve) for the network topology of flows at each instant.  The same computation can be done for the rows of C ij thus profiling the entropy structure for flows away from each node.

Mathematical Overview - 3  The row and column entropy spectral curves can be computed and monitored in real time for any network topology and thus compared to the ‘normal’ profile.  Deviations from the norm can be visually identified or by statistical correlation or by wavelet expansions.  Deviations can be instantly tracked to the exact set of causative nodes for action.

End of Technical Overview  An extensive analysis on the methodology is available at  Thank You for your interest.