ExaSphere Network Analysis Engine © 2006 Joseph E. Johnson, PhD

The Network Problem  Networks are vast and complex contain astronomical amounts of data  Yet there has been no method of tracking their behavior. Until NOW

What is a Network?  A network is a set of points (or nodes) with some of the points connected.  Examples: Airline networks – airports connected by the number of flights or passengers. Internet networks – computers connected by the amount of information sent. Financial networks – bank accounts connected by the amount of fund transfer.

ExaSphere: Origin & Purpose  The ExaSphere Network Engine is the product of years of research funded at over $2.5M by DARPA (US Dept. of Defense).  The ExaSphere Network Engine is a general purpose software solution that can be applied to any network for tracking and analysis.

Why is the Problem Difficult?  Networks contain millions of nodes and trillions of changing values.  Yet all values are equally important. Thus the quantity of data is overwhelming.  Networks are among the most difficult of generally unsolved mathematical problems.  We need to reduce this vast data to a few representative values – called network metrics.

Where can this software be applied?  Communications Networks Internet Traffic: malicious process detection Phone (land & wireless): system behavior Mail (UPS, FedEx..): flow monitoring  Transportation: patterns & demand Air Traffic Roadways (personal & trucking) Railroads Pipelines International Shipping

Also:  Financial Banking Transactions Monitoring Accounting Flows Ownership & Investments  Utility & Energy Electrical Grids Water & Sewer Flows Natural Gas & Petroleum Distribution

And Also:  Social Networks Organizational & Business Criminal & Terrorist Networks  Manufacturing Process Supply Chain Manufacturing Workflow  Health & Biological Disease Ecological

How Does it Work - 1  The data input has four parts in each record: (t, i, j, w) 1. Date-Time of the action (t) 2. The originating node identifier (i) 3. The receiving node identifier (j) 4. The weight or value of the connection (w)  These weights are added into a matrix, C ij over an interval of time (minute, day,…)

How Does it Work - 2  New mathematical discoveries are invoked:  The ‘connection matrix’ C ij is processed to a set of values that represent the order and disorder (entropy metrics) of the connectivity structure for each node.  These metrics are tracked as a curve in real time that reveals abnormal changes and at which nodes they occur.

Natural Time Frames:  Internet Traffic and Electrical Grids must be rapidly monitored perhaps every minute.  Banking Transactions have a natural time of perhaps a day as would most Transportation Flows while Accounting Flows have a more natural time of one month.  Criminal Networks would probably build a more permanent connection structure.

Expected Changes  Certain structural changes are natural to different networks and such behavior has to be considered to identify anomalies: Internet, telephone, and transportation flows depend upon the time of day and day of the week. Electrical grids are highly dependent upon the weather as well as the time of day.

The ExaSphere Network Analysis Engine  - is the first and only completely generalized tool for network monitoring  - tracks the entropy spectra over time identifying abnormal behaviors, attacks, and system failures, by node  - is a general purpose software engine solution for all types of networks

Software Environment  The core software components have been built in both professional JAVA and Mathematica languages independently.  Current work: Expand the user tools. Develop success stories for different network applications. Continue our analysis of internet behavior at two sites.

End of Business Overview  Detailed information is available at

Mathematical Overview -1  The inventor discovered that every possible network structure, C ij, corresponds to exactly one infinitesimal generator of a Markov transformation.  Each such Markov transformation consists of columns that are valid probability distributions (non-negative values normed to unity).

Mathematical Overview - 2  Each such column vector represents a transformation probability profile toward a node.  Shannon and generalized Renyi entropy functions can then be rapidly computed on each column giving the order / disorder of the probability profile for that node.  These entropy metrics can be sorted in order to represent the entropy spectra (a curve) for the network topology of flows at each instant.  The same computation can be done for the rows of C ij thus profiling the entropy structure for flows away from each node.

Mathematical Overview - 3  The row and column entropy spectral curves can be computed and monitored in real time for any network topology and thus compared to the ‘normal’ profile.  Deviations from the norm can be visually identified or by statistical correlation or by wavelet expansions.  Deviations can be instantly tracked to the exact set of causative nodes for action.

End of Technical Overview  An extensive analysis on the methodology is available at  Thank You for your interest.