Learning-Based Anomaly Detection in BGP Updates Jian Zhang Jennifer Rexford Joan Feigenbaum.

Slides:

Advertisements

Similar presentations

Applications of one-class classification

Advertisements

Ziming Zhang, Yucheng Zhao and Yiwen Wan.  Introduction&Motivation  Problem Statement  Paper Summeries  Discussion and Conclusions.

1 Copyright by Jiawei Han, modified by Charles Ling for cs411a/538a Data Mining and Data Warehousing v Introduction v Data warehousing and OLAP for data.

Ch2 Data Preprocessing part3 Dr. Bernard Chen Ph.D. University of Central Arkansas Fall 2009.

Comparing IPv4 and IPv6 from the perspective of BGP dynamic activity Geoff Huston APNIC February 2012.

November 12, 2013Computer Vision Lecture 12: Texture 1Signature Another popular method of representing shape is called the signature. In order to compute.

UC Berkeley Online System Problem Detection by Mining Console Logs Wei Xu* Ling Huang † Armando Fox* David Patterson* Michael Jordan* *UC Berkeley † Intel.

Detection of Deviant Behavior From Agent Traces Boštjan Kaluža Department of Intelligent Systems, Jožef Stefan Institute Jozef Stefan Institute Jožef Stefan.

1 BGP Anomaly Detection in an ISP Jian Wu (U. Michigan) Z. Morley Mao (U. Michigan) Jennifer Rexford (Princeton) Jia Wang (AT&T Labs)

Statistical Approaches for Finding Bugs in Large-Scale Parallel Systems Leonardo R. Bachega.

Sensitivity of PCA for Traffic Anomaly Detection Evaluating the robustness of current best practices Haakon Ringberg 1, Augustin Soule 2, Jennifer Rexford.

1 Measurement of Highly Active Prefixes in BGP Ricardo V. Oliveira, Rafit Izhak-Ratzin, Beichuan Zhang, Lixia Zhang GLOBECOM’05.

Lecture 6: Hybrid Robot Control Gal A. Kaminka Introduction to Robots and Multi-Robot Systems Agents in Physical and Virtual Environments.

BGP Safety with Spurious Updates Martin Suchara in collaboration with: Alex Fabrikant and Jennifer Rexford IEEE INFOCOM April 14, 2011.

DYNAMICS OF PREFIX USAGE AT AN EDGE ROUTER Kaustubh Gadkari, Dan Massey and Christos Papadopoulos 1.

Incrementally Deployable Security for Interdomain Routing (TTA-4, Type-I) Jennifer Rexford, Princeton University Joan Feigenbaum, Yale University January.

1 Finding a Needle in a Haystack: Pinpointing Significant BGP Routing Changes in an IP Network Jian Wu (University of Michigan) Z. Morley Mao (University.

Self-Correlating Predictive Information Tracking for Large-Scale Production Systems Zhao, Tan, Gong, Gu, Wambolt Presented by: Andrew Hahn.

Multi-Scale Analysis for Network Traffic Prediction and Anomaly Detection Ling Huang Joint work with Anthony Joseph and Nina Taft January, 2005.

Basic Concepts and Definitions Vector and Function Space. A finite or an infinite dimensional linear vector/function space described with set of non-unique.

University at BuffaloThe State University of New York WaveCluster A multi-resolution clustering approach qApply wavelet transformation to the feature space.

Patricio Vielva Astrophysics Department (IFCA, Santander) Currently Astrophysics Group (Cavendish Lab., Cambridge) Wiaux, Vielva, Martínez-González.

Review Dec, 2001 Workpackage 4 Image Analysis Algorithms Progress Update Dec Kirk Martinez, Paul Lewis, David Duplaw, Fazly Abbas, Faizal Fauzi,

02/06/2006ecs236 winter Intrusion Detection ecs236 Winter 2006: Intrusion Detection #4: Anomaly Detection for Internet Routing Dr. S. Felix Wu Computer.

School of Computer Science and Information Systems

EL 933 Final Project Presentation Combining Filtering and Statistical Methods for Anomaly Detection Augustin Soule Kav´e SalamatianNina Taft.

Root cause analysis of BGP routing dynamics Matt Caesar, Lakshmi Subramanian, Randy H. Katz.

Algorithm: For all e E t, define X e = {w e if e G t, 1 - w e otherwise}. Measure likelihood of substructure S by. Flag S as anomalous if, where is an.

Video Trails: Representing and Visualizing Structure in Video Sequences Vikrant Kobla David Doermann Christos Faloutsos.

FIB Aggregation Zartash Uzmi draft-uzmi-smalta-01 (with Ahsan Tariq and Paul Francis)

Intrusion and Anomaly Detection in Network Traffic Streams: Checking and Machine Learning Approaches ONR MURI area: High Confidence Real-Time Misuse and.

Data Mining for Intrusion Detection: A Critical Review Klaus Julisch From: Applications of data Mining in Computer Security (Eds. D. Barabara and S. Jajodia)

A Geometric Framework for Unsupervised Anomaly Detection: Detecting Intrusions in Unlabeled Data Authors: Eleazar Eskin, Andrew Arnold, Michael Prerau,

Intrusion Detection Jie Lin. Outline Introduction A Frame for Intrusion Detection System Intrusion Detection Techniques Ideas for Improving Intrusion.

Describing and Exploring Data Initial Data Analysis.

1 A Bayesian Method for Guessing the Extreme Values in a Data Set Mingxi Wu, Chris Jermaine University of Florida September 2007.

Understanding the Network-Level Behavior of Spammers Best Student Paper, ACM Sigcomm 2006 Anirudh Ramachandran and Nick Feamster Ye Wang (sando)

High-Level Design With Sequence Diagrams COMP314 (based on original slides by Mark Hall)

Using Identity Credential Usage Logs to Detect Anomalous Service Accesses Daisuke Mashima Dr. Mustaque Ahamad College of Computing Georgia Institute of.

Scalable Analysis of Distributed Workflow Traces Daniel K. Gunter and Brian Tierney Distributed Systems Department Lawrence Berkeley National Laboratory.

Time Series Data Analysis - I Yaji Sripada. Dept. of Computing Science, University of Aberdeen2 In this lecture you learn What are Time Series? How to.

A Framework for Elastic Execution of Existing MPI Programs Aarthi Raveendran Graduate Student Department Of CSE 1.

Automated Problem Diagnosis for Production Systems Soila P. Kavulya Scott Daniels (AT&T), Kaustubh Joshi (AT&T), Matti Hiltunen (AT&T), Rajeev Gandhi (CMU),

INTERACTIVE ANALYSIS OF COMPUTER CRIMES PRESENTED FOR CS-689 ON 10/12/2000 BY NAGAKALYANA ESKALA.

Interdomain Routing Security. How Secure are BGP Security Protocols? Some strange assumptions? – Focused on attracting traffic from as many Ases as possible.

BGP topics to be discussed in the next few weeks: –Excessive route update –Routing instability –BGP policy issues –BGP route slow convergence problem –Interaction.

Detection of Routing Loops and Analysis of Its Causes Sue Moon Dept. of Computer Science KAIST Joint work with Urs Hengartner, Ashwin Sridharan, Richard.

SIGCOMM 2012 (August 16, 2012) Private and Verifiable Interdomain Routing Decisions Mingchen Zhao * Wenchao Zhou * Alexander Gurney * Andreas Haeberlen.

: An alternative representation of level of significance. - normal distribution applies. - α level of significance (e.g. 5% in two tails) determines the.

Generating Software Documentation in Use Case Maps from Filtered Execution Traces Edna Braun, Daniel Amyot, Timothy Lethbridge University of Ottawa, Canada.

BotCop: An Online Botnet Traffic Classifier 鍾錫山 Jan. 4, 2010.

© Tan,Steinbach, Kumar Introduction to Data Mining 4/18/ Data Mining: Data Lecture Notes for Chapter 2 Introduction to Data Mining by Tan, Steinbach,

Sensitivity of PCA for Traffic Anomaly Detection Evaluating the robustness of current best practices Haakon Ringberg 1, Augustin Soule 2, Jennifer Rexford.

Bringing External Connectivity and Experimenters to GENI Nick Feamster Georgia Tech.

Anomaly Detection. Network Intrusion Detection Techniques. Ştefan-Iulian Handra Dept. of Computer Science Polytechnic University of Timișoara June 2010.

1 Visualizing Multi-dimensional Clusters, Trends, and Outliers using Star Coordinates Author : Eser Kandogan Reporter : Tze Ho-Lin 2007/5/9 SIGKDD, 2001.

3/13/2016Data Mining 1 Lecture 1-2 Data and Data Preparation Phayung Meesad, Ph.D. King Mongkut’s University of Technology North Bangkok (KMUTNB) Bangkok.

Distributed Network Monitoring in the Wisconsin Advanced Internet Lab Paul Barford Computer Science Department University of Wisconsin – Madison Spring,

Kernel Based Anomaly Detection Andrew Arnold (aoa5) 2nd Annual Project Student Day Columbia University -- 4/26/01 Intrusion Detection Systems -- IDS Machine.

Clickprints on the Web: Are there Signatures in Web Browsing Data?

Jian Wu (University of Michigan)

BGP-lens: Patterns and Anomalies in Internet Routing Updates

The Functional Space of an Activity Ashok Veeraraghavan , Rama Chellappa, Amit Roy-Chowdhury Avinash Ravichandran.

Intermittency and clustering in a system of self-driven particles

Statistical Inference for the Mean Confidence Interval

Data Preprocessing Copyright, 1996 © Dale Carnegie & Associates, Inc.

Anomaly Detection in Crowded Scenes

Data Transformations targeted at minimizing experimental variance

Data Pre-processing Lecture Notes for Chapter 2

Can Genetic Programming Do Manifold Learning Too?

Presentation transcript:

Learning-Based Anomaly Detection in BGP Updates Jian Zhang Jennifer Rexford Joan Feigenbaum

Aug. 26, Motivations Identifying anomalous BGP-updates is important. Detecting security problems Flaky equipment It’s hard to define “anomalies.” Only know the signatures of a few types of anomalies (e.g., constant updating) Still at an early stage in understanding: What are the anomalies? What signal they generate?

Aug. 26, Anomalies in Update Dynamics Anomalies in update dynamics may reflect anomalies in the BGP updates. From a router’s view, update dynamics show as a sequence of update messages. Temporal features of this sequence are important in anomaly detection. Message burst duration and intensity Inter-burst interval

Aug. 26, Previous Analyses of Update Dynamics Many use simple aggregations. Consider aggregations over time interval T. Temporal features at levels finer than T are lost. To detect constant updating, these features may not be necessary. They may be needed to identify other types of anomalies. Some suffer from the magic number problem.

Aug. 26, Our Approach Learn a “model” of “normal” update behavior. Identify updates that deviate significantly for further investigation. Difference from previous work: Multi-scale analysis Representation captures more temporal features.

Aug. 26, Transformation of Update Message Signals We view the sequence of update messages for each prefix as a signal along time: Apply a wavelet transformation to the signal to reveal its temporal features. Time # of Messages

Aug. 26, Representation of Update Dynamics Build histograms for the distributions of the temporal features. View the histograms as a vector. A trace of update dynamics becomes a point in a vector space. The transformation and the representation capture temporal features at different time scales.

Aug. 26, Avoid Magic Numbers It is hard to determine a good value for the magic numbers. We consider a set of values in an interval [T min, T max ]. Using an interval large enough, our analysis can avoid the magic-number problem.

Aug. 26, Clustering Traces of update dynamics are mapped into points in a vector space. Clustering groups the update dynamics into clusters to reveal different types of dynamics.

Aug. 26, Learn Normal Dynamics Normal dynamics: regions containing most of the update traces Abnormal dynamics: traces mapped to a location far away from the normal

Aug. 26, System Overview Signal of updates Wavelet transformation Distribution of message-burst durations and intervals Representation in a vector space Learn normal dynamics and detect anomalies

Aug. 26, Experiments RouteViews data 6 Months of update messages Combined update messages from all RouteViews vantage points. Clustering for a single prefix along time and across prefixes.

Aug. 26, Preliminary Results Focusing on individual prefixes: Typically, the largest cluster contains 80-90% of instances of the update dynamics. Across prefixes: Several (3-4) largest clusters contain about 50% of the prefixes. In both cases, constant updating show as outliers.

Aug. 26, Further Investigation Ongoing work to find out: What are the particular types of dynamics in each cluster? Are the updates in the small clusters that deviate from the normal real anomalies? Use labeled examples to build the knowledge base. Incorporate the route attributes in our representation.