Network Asset Management at Jefferson Lab Bryan Hess, Andy Kowalski, Brent Morris,

Topics Network redesign & segmentation The management system hardware & software The end user experience The help desk staff experience Next steps

Motivation & Goals 1.Network Segmentation: To enforce that only those machines that need to communicate can. 2.Admission Control: To ensure that the networks stays segmented 3.Registration: To know who is in charge of each machine 4.Reporting: To be able to know the state of the network 5.Management Console: Simplify Adds/Moves/Changes with a web interface

Segmentation & Network Redesign

Segmentation: Design Move away from per-building non-firewalled networks Create vlans for machines based on their purpose and security profile. Examples: –Centrally managed desktop machines –User managed machines –Farm nodes –Data acquisition –… New IP addresses for every host on legacy building networks. Use firewalls between vlans to enforce who talks to whom

Segmentation: Firewall rules Group related vlans into Cisco FWSM contexts Implement most access control rules on the “inbound” side from router to vlan keep rules affecting network X on the inbound side of network X as much as possible

Segmentation: Scientific Systems For high throughput networks firewalls are not sufficient –We have no 10Gbit firewalls –We use a similar strategy for segmentation, but with simple router-based ACLs No direct internet access for these systems; Some web proxy access Avoid changing the way these complex existing systems work, but insulate them as much as possible

Segmentation: Admission Control Network Segmentation requires enforcement Must ensure that a given MAC address stays exclusively on its assigned network Port Security used an interim solution while we were “sorting” machines into vlans. Big headache –users are caught unaware by port security –Easy to make a mistake during moves

Segmentation: 802.1x MAB The real solution: switch port that change vlan assignments dynamically based on the MAC address connected 802.1x MAC Authentication Bypass (MAB) solves this problem nicely Cisco support for MAB is improving. Switch contacts a RADIUS server (backed by our database) to get its vlan assignment.

Segmentation: Auto-vlan assignment We call this use of 802.1x MAB “auto-vlan assignment” We have it in use in every office space on site and most lab spaces. We do not use it for data centers or embedded/data acquisition settings. Auto-vlan ports are authenticated based on MAC address when they connect. This is largely transparent to users. Moving to another network jack “just works” in many cases

The management system

Management System Hardware 802.1x capable switches –Cisco 2960, 4500, 6500 series MySQL database –Reliable hardware (RAID, N+1 power) –Live replica on a backup machine Redundant RADIUS servers Web servers on VMWare ESX cluster Why so much redundancy? 802.1x. If the system is down, machines are not admitted to the network as they are connected.

JLab developed Software Perl and PHP Monitoring dæmons & database back end (Gator) –SNMP monitoring of switches and routers –Aggregated into a single database –Wiring, Registration PHP Front End (Jnet) –Lots of scripts to glue everything together –DNS –DHCP –Switch configuration –Machine registration –Database queries and reporting

User Experience: New Machines Newly connected machines go to a “limbo” network where they can only access a registration web page This page requires a login, so it collects username and MAC address automatically. Final VLAN assignment is made by staff.

Management: Machine assignments Combines into one web page: DHCP, DNS, vlan assignment, and registration. Add/Move/Change process is greatly streamlined Many checks to avoid duplicates or errors All-or-nothing changes

Management: A change is made

Finding Machine Registrations

Finding a Switch

Switch View with Wiring, Registration

Room View: Wiring and Machines

Searches: Historical Information Recorded history of interesting associations MAC/IP MAC/VLAN MAC/Switch Port This data is very useful in conjunction with the wiring database Also used by –Missing Property –Cyber Security

Successes The network is segmented & firewalled We routinely locate the physical location of a machine based on owner, ip address, mac address, property tag, or host name. JNet prevents mismatches between DNS, DHCP, and vlan assignment. Add/Move/Change requests are trivial since all office space uses auto vlan assignment Network-related help desk requests are down

Next Steps: Host-based Introduction of host-based monitoring –admission control to fix the “Hey, that Linux machine was running XP last week” problem. –An agent on the machine? –External security monitoring Create a “Penalty Box” network for remediation –Quarantine Machines as needed –Allow them to patch or rebuild –Provide web notification that the machine is quarantined

Next Steps: Wireless Wireless networks are a different can of worms We currently do user-based authentication –Allows unvetted machines on the network Need to do machine-based (MAB-style) authentication to make wireless more like wired Moving to a different wireless solution to do this.

Questions? Bryan Hess Andy Kowalski Brent Morris