1. Network Data Collection Infrastructure to Detect Security Anomalies
Gen Ohta and Alina Oprea, Northeastern University, Boston, USA
Overview
Goals
Determine the performance impact of deploying sFlow/NetFlow data collection in MOC
Set up a local staging environment to measure performance impact of network traffic collection
Deploy sFlow data collection on Brocade fabric in Engage1 environment
Use machine learning algorithms to detect security anomalies and improve cloud security
Overview
Staging Environment
MongoDB Schema
Engage1 Environment
Analytics

2. Network Data Collection Infrastructure to Detect Security Anomalies
Gen Ohta and Alina Oprea, Northeastern University, Boston, USA
Staging Environment
Experiment
Goal
Determine performance impact of deploying sFlow data collection in MOC
Metrics
Latency - delay between when a request is issued and when is completed
Throughput - number bytes per unit time transferred
Control
Issue requests from traffic generator to Apache HTTP server
Record latency and throughput values
Repeat steps 1 and 2 for 10 times
Increment number of requests and repeat until maximum is reached
Test
Set sampling rate and polling interval on Brocade VDX switch
Run control experiment and repeat for different parameters
Determine optimal configuration of switch parameters
Overview
Staging Environment
MongoDB Schema
Engage1 Environment
Analytics

3. Network Data Collection Infrastructure to Detect Security Anomalies
Gen Ohta and Alina Oprea, Northeastern University, Boston, USA
MongoDB Schema
Overview
Staging Environment
Persistent storage
Internal and external flows
Persist data in NoSQL database
Common fields for sFlow/NetFlow
Can index on multiple fields
MongoDB Schema
Engage1 Environment
Analytics

4. Network Data Collection Infrastructure to Detect Security Anomalies
Gen Ohta and Alina Oprea, Northeastern University, Boston, USA
Engage1 Environment
Overview
Staging Environment
Data Collection
Deploy data collection on all switches in Brocade fabric in Engage 1 HPC cluster
Deploy Brocade flow collector on multiple servers
Create MongoDB cluster to store sFlow data
MongoDB Schema
Engage1 Environment
Analytics

5. Network Data Collection Infrastructure to Detect Security Anomalies
Gen Ohta and Alina Oprea, Northeastern University, Boston, USA
Analytics for security applications
Use cases
Detect suspicious communication with external IP addresses
Detect data exfiltration attempts
Prevent DDoS attacks
Prevent cloud abuse
malware infection, application exploits, illegal use of cloud resources
Techniques
Graph modeling of internal and external communication patterns
Correlate with performance metrics collected by monitoring team
CPU, I/O, memory, power
Machine learning algorithms: clustering, graph propagation, outlier detection, time-series anomaly detection
Overview
Staging Environment
MongoDB Schema
Engage1 Environment
Analytics