Frequency Analysis of Protocols Dr. Craig Partridge BBN Technologies

An Emerging Field Using techniques from signal processing to better understand networks and protocols A quick tour of the work done to date Along with some highly speculative thoughts about what might come next

An Overview of the Basic Concepts Please note, I’m a systems person, not a mathematician. This talk structured for an intuitive understanding… … although I’ll try to be rigorous where necessary

Step 1: Capture Packet Traces Place taps or measuring devices in various spots in the network For each transmission seen, capture Time Direction Duration Other stuff as desired Network tap

Step 2: Trace to Signal Trace is a discrete time series (time + data in non-uniform time increments) Signal processing wants a time/amplitude series (often a uniform series)

Step 3: Run Feature Detection Algorithms over Signal The meat of the task…. Indeed, the signal representation you chose is largely dictated by the algorithm you wish to run Various algorithms extract various types of information Rest of the talk is a survey of what has been done

USC DDoS Attack How many sources are attacking you? Capture attack packets Convert to a uniform series x(t) = # of attack packets received in millisecond t Condition signal Subtract mean x(t) out Removes dominant frequency

DDoS Continued Now do auto-correlation and compute spectral density Basically looking for frequency variations in the attack stream over time A uniform source would show a single stable set of frequencies Spectral-density: a spectrum where you show the power at each frequency

Wavelet-based Approach Huang, Feldmann, Willinger Finding time structures in traces Capture packet traces at some point Divide into conversations/flows Use source/destination/prefix info to do division Divide according to what class of traffic you wish to analyze Convert traces to uniform signal of 0/1

More Wavelet Compute an energy function Compute discrete Haar wavelet transform Energy function measures wavelet coefficients Low coefficients reveal regular or periodic structure in time series Use energy graphs to reveal periodic structure

Lomb Periodogram Cousins, Krishnan, Partridge Similar to wavelet approach Lomb periodogram: designed for non-uniform signal traces [ideal for packets] Computes spectral power at each frequency

Lomb Example

Example Results Identified CBR Send Rates Identified FTP Round Trip Times Characteristics from all three flows observed

Node ID Application Frequencies (Hz) XX0– X001.0 X X X031.0 X041.0 X X061.0 X071.0 X X091.0 X X X121.0 Xp Xp Xp Xp Xp Xp Green: Correct Detection Red: Missed Detection Data: 18 nodes, tcpdump Results: Detected 6 out of 6 application frequencies emitted Detected 15 out of 27 traffic generators Missed most generators emitting at 1 Hz Spectral Techniques easily show periodic application traffic on the network Lomb Analysis of b Data Hz

A Pause to Comment All three approaches mentioned so far have the characteristic that We can detect timing structure from our data If we have ground-truth, we can show how the timing structure we find relates to the timing structures in the network But, without ground-truth, we can’t say for sure what the structure means

Topology Discovery Techniques where we can show a valuable set of results, without ground-truth to interpret Discover links in a network (wireless) Coherence Causality Given complete map, which links are used? Route discovery

Coherence Take samples of the time series at different points in the network Compare them, offset in time Look for statistically significant relationships between their spectral peaks

A Sketch of the Coherence Math Compute the Discrete Fourier Transform This gives you a series of equally spaced points in a spectrum The Cross Spectral Density is an averaged product (for each of the points in the spectrum) of the DFT of one series with the complex conjugate of the DFT from another series Normalize the CSD to 0…1 to get coherence

Coherence Plots

Coherence Comments Coherence works Nicely tracks moving nodes But coherence gets confused For instance, confusion over applications with similar periodicity Sometimes skips hop in path

Causality Instead of related spectra, try relating individual transmissions to transmissions that came before Define a weight function W that estimates the likelihood that event k came from a prior transmission by node i Then the probability that an event at node i caused k is:

Topology Discovery Now create a conversation matrix Consider C which is the set of all events at a particular node i. The probability that node j is sending to node i is: These values define a matrix Row x is probabilities that x is sending to each of the nodes Column x is probability that x is receiving from each of the nodes N.B.: Probability can be computed incrementally over C

Comments on Causality Core idea: Over the course of a number of events, the probability function will give enough more weight to correct sources to yield a good conversation matrix Current W is pretty simple Exponential (Poisson) focused on most recent event Self similarity not a problem until we look fairly deep back in time May need a more expensive weight function Very fast… (real time analysis)

Egress Nodes Extend the causality equation For each event, compute 1 minus maximum weight: the egress weight I.e. figure weighting algorithm correctly identified source of event, if present. If no source, this inverse will be large Define a new column of the conversation matrix that contains the normalized average of the egress weight. Large values flag egress nodes

Egress Example

Stitching Once egress nodes identified, it is possible to connect graphs efficiently Each probe shares with its neighboring probes the traffic traces from its egress nodes Traces are combined to create a single trace between each set of pairs Rerun the topology algorithm with the additional trace and see if a link appears

Stitching Example

Thoughts on Egress and Stitching Extensions to causality analysis Egress is highly dependent on the weighting function

End-to-End Route Discovery Discover end-to-end paths between communicating hosts (src and dst) Route: A path or sequence of links (src to dst) There may be multiple paths – need the path actually taken by data from src to dst Require identification of active links Can do receiver identification from conversation matrix Choose shortest paths Break ties using “aggregate path coherence” Coherence between steps in each path End result: Layer 3 (network) connectivity – Routing Tables

Some Thoughts Progress is likely to be rapid Better techniques Match and latch Max-plus Timing structure is remarkably robust E.g. Lomb showed frequency of traffic that wasn’t visible