Shared Data Access Network (SDAN) for Monitoring, Security, Performance J. Scott Haugdahl Principal Engineer, Blue Cross Blue Shield Former Asst. VP & Architect, US Bank Data Connectors Minneapolis, March 28th, 2013

The US Bank Experience Who is US Bank (Symbol: USB)? Part of U.S., a diversified financial services, holding company Fifth-largest commercial bank in the U.S with over 3,000 branches Recognized for its strong financial performance and prudent risk management, capital generation, and product quality What is Network Application Analysis (NAA)? Founded in 2008 as part of US Bank’s Network Planning and Engineering to adapt new thinking methods, tools, process, and collaboration in order to focus on resolving potential or chronic application performance problems Solutions oriented, not only the lower network (i.e. infrastructure) layers Gained a high level of visibility and credibility during pre-migration analysis to new data center Created the Shared Data Access Network (SDAN) to support security, monitoring, and analysis tools Why the SDAN? The only solution able to collect and aggregate multiple streams simultaneously from several tiers in real-time to feed Application Performance Monitoring (APM), fraud detection, security, and sniffer tools

Oh oh! Now what do we do?!

The Dark Ages “Technicians had to physically unplug and move tools from one tap or SPAN port to another. That necessitated change orders and scheduling during off hours, slowing the group’s agility and flexibility to monitor effectively.” - Royal Bank of Canada The data center crash cart. Not to mention tripping over wires or pulling the wrong one.

Is This the Best We Can Do?

Sharing SPANs Got Ugly Hey, It’s MY SPAN PORT! (Referee from Gigamon) (Dropped Packets) (Blade Server) Only two ports per 6509, problem with oversubscribing, port channels, Nexus 7k with top of rack 2k’s (line card fabric extension) is collapsing the architecture and making it worse – still only 2 spans per 7k

Fast Forward Gigamon Intelligent Matrix The Shared Data Access Network (SDAN) Collects & Sends Packets to Consumers Tapped Media Mirror Ports Load Balancers Firewalls Mainframe Switches UCS Fabric Blade Chassis Packet Sources Gigamon Intelligent Matrix Switching, Filtering, Aggregation, Slicing, etc. - SDAN – another name for Gigamon’s “Visibility Fabric”. The scope of the SDAN is the taps and matrix switches… all other devices are consumers of the SDAN, a very important distinction to make when “selling it”. Sniffers can assist in DDoS and IDS leakage analysis. Intrusion Detection Fraud Threat Analysis Data Loss Prevention APM Sniffer Consumers

SDAN Value – The Big Three  Collect and Aggregate Packet Flows Several streams from multiple tiers can be collected and aggregated to one or more 10 Gbps outputs, in order to monitor complex applications and save on tool ports  Passively Share Packet Flows Packet stream sources (network ports) can service many consumers (tool ports) critical to protecting your customers and improving the end-user experience This really is the only solution that can effectively collect multiple packet streams and aggregate them to out-of band tools.  Filter and Preprocess Packet Flows Flows can be filtered by MAC, VLAN, IP (and sliced, de-duped, etc.) allowing focused analysis or fraud detection and significant drop in CPU demand on the tool or appliance

Simplified App Mapping & Tapping Application “X” Internet Users Tier 3 Tier 1 Load Balancer Authentication Internet Routers “X” Web Servers Policies Load Balancer “X” App Servers “X” DB Servers Firewalls Tier 2 Load Balancer “DMZ” Tapping above and below load balancers are great places to pick up services to monitor, isolate faults by domain, troubleshoot, optimize apps Load Balancer Messaging Access GW Mainframe Firewalls

Steps to a Successful SDAN Deployment Document the logical flow of the application In complex environments, use application (not network) conceptual flow diagrams to determine the logical tap points per end-tool requirements (packet analysis, security, APM, etc.) Different applications will have different flows and services, especially customer facing vs. internal applications Map the logical flows and devices to physical ports Example: Firewalls and where they attach Tap the physical media into your SDAN network ports These comprise the ingress or network ports Aggregate the packet streams and send to your SDAN tool ports Filters may be required to remove irrelevant packets Feed the security flows to your sniffer to validate your setup Don’t forget this important last step! IDS security below the firewall is a given. But what about fraud detection, data loss prevention, and other such tools? Validate your packet flows through the SDAN before an attack or breach!

After SDAN With the SDAN, we are now one big happy family sharing the sandbox! Note the Gigamon orange color of the sandbox. :) 

Some SDAN Security Tool Best Practices Tap related network points into a Gigamon 420 or TA1 and send aggregated flows to 2404/HD4/HD8 for security tool consumption Example: Tier 1 Firewalls and Interfaces -> TA1 -> Tier 1 Firewall Aggregate -> HD8 -> IDS Example: Nexus 2232/2248’s -> TA1 -> Server Farm Aggregate -> HD8 -> Fraud Detection Example: Mainframe OSAs -> TA1 -> Mainframe Aggregate -> HD8 -> Data Loss Prevention Use rules and filtering to greatly reduce load on the security appliance Security and APM appliances do not need to waste cycles filtering irrelevant data Reducing unnecessary intake can also increase post analysis processing performance SPANs (and mirror ports) usefulness is diminishing, so avoid if possible Easy to over subscribe, especially with port channel or full duplex aggregation Eliminate the old practice of using aggregation taps and use fiber where possible Be mindful that each tap requires two SDAN ports when operating in non-aggregation mode Consider preserving separate send/receive full duplex tap ports all the way through to your tools for certain data center or branch WAN connections Preserving full duplex tapped router connections helps to preserve incoming vs. outgoing Copy your security flows to permanent sniffers for post mortem analysis Data mine stored packet flows for deep dive forensics analysis - Preserving the send and recieve side of full duplex taps to our tools can help reserve send/receive reports and statistics

Not Best Practices!

Thank You!