1. SLAC’s Networks Prepared by: Les CottrellSLAC,
The background topology map was created by the network monitoring topology feature.
Prepared by: Les CottrellSLAC,
for SLAC Network & Telecommunications groups
Presented to Kimberley Clarke
March 8th 2011

2. Outline Phone upgrade Core network & offsite connections
Cell phone coverage, mobility
Wireless, visitor subnet
Monitoring LAN & WAN
Gigamon
VPN upgrade
IPv6, IPAM
Conclusions

3. Philosophy Support getting the science done (safely)
The science is the mission
Uniformity of design (where possible)
Define standardized solutions & apply repeatedly
Limit vendors, technologies used
Leverage existing OCIO staff expertise
Engineered for robustness (e.g. redundancy)
OCIO is not staffed for 24/7 coverage
“Throwing smart (dedicated) people at issues” works as long as you do not throw them too often
Powerful, easy to use monitoring
Network support staff are:
3 FTE in operations who do installs, cabling, terminations, physical trouble shooting
2.5 SLAC FTEs in telecom plus one contractor
4 FTEs (currently -1) in network engineering including design, 2nd level troubleshooting, addressing, routers/switch/firewall configurations etc.

4. Central phone system
Designed for low cost ($15/phone/month) , high reliability (1 unscheduled system fail in 22 years – loss power)
End of life: parts are 1988 vintage, last major update 2000
4000 phones, ~ 50% are non user (e.g. wall, conference room, FAX, emergency …, so can stay analog)
Evolutionary upgrade phone system using existing infrastructure (phone sets, closets, UPS, cabling) where possible to reduce costs and ensure maintainability while we:
Enable VoIP
Enable unified communications
/vmail integration, presence, mobility, SMS …
Cost/phone/month <$15/month. Cheapest in DoE laboratories. One uncsheduled system outage in 22 years.

5. Network Scale 70 major buildings,
Single site, but lots of worldwide collaborations
300 layer 2 capable devices, 50 layer 3
15K end devices, 30K ports,
Support:
science (open high performance worldwide),
business (protected, e.g. HR, finances ..),
controls & monitoring systems (local HVAC, accelerator),
desktops with local & internet access
visitors
SLAC site 440 acres ~ 250 buildings, 2000 people on site including SLAC employees, users/visitors, contractors etc.

6. Farm = science high performance cluster

7. Local Area network
Core network: highly reliable, supports 10Gbps connections for:
high performance computing clusters, offsite, and buildings (edge) switches,
Redundancy for power, routers, power supplies etc.
Most wired desktops can be/are enabled for 100Mbps connections, we are upgrading to 1Gbps to the desktop for major buildings.
Segmenting and rationalizing subnets
Private (RFC1918), Internet access, printers
Subnet set/switch, removing flat earth
Improved security, isolation of problems & performance
Flat earth = very large layer 2 domains (trunked VLANs) extending across multiple buildings.
Because of pricing and other issues, most observers don't expect 100GE to be widely used until 2013 in service providers and at least 2015 in data centers.

8. Accelerator Control network
The SLAC LINAC is operated via an IP based control network.
About 4 mile long, about 80 individual network switches, 4000 switch ports
Routed centrally, dual redundant routers and links to each switch
Uses IP multicast technology for real-time feedback and control at 120Hz
Deterministic latency design: all traffic for each pulse must be delivered within 1ms
Centrally designed and maintained: the entire network is based on only two platforms: Cisco 6509 for core routing and switching, and stackable Cisco 3750G switches for access.

9. Wide Area Network Access
Off site links: multi 10Gbps links
ESnet most production and also dedicated circuits (using MPLS) to BNL for ATLAS
Stanford and CENIC/Internet2
One physical path down Sand Hill Rd AT&T conduits with IRU
SRCF 2nd redundant path
ACLs at borders
Picture shows ESnet Juniper router
SRCF = Stanford Research Computing facility (break ground end 2011)
Do not fully firewall at border (only ACLs) due to costs and performance impacts

10. Mobility WiFi: most buildings covered ~ 160WAPs
Open access, not authenticated: ease of use
No privileged access to SLAC resources
Visitor subnet: no servers, block inbound connections

11. Cell phones
Coverage outside good: on site macro sites for T-Mobile, Sprint, Metro-PCS and AT&T. Verizon going in across the street
In buildings: most are penetrated from outside.
Installed BDAs in a few heavily shielded buildings
Pico cell in one area
Pagers at end of life (atrophied ’60s technology)
Accelerator tunnel communications is a concern. Current pagers beyond end of life. Looking at alternatives to support cellphones and/or pagers and/or WiFi

12. Monitoring Critical enabler for network and desktop admins
LAN: lookup routers, switches, ports, hosts, hosts for person, MAC & IP addresses, VLANs, provide:
History, uilization, temp, cpu, power use, weather maps, idle ports, topology
WAN: collaborations worldwide, E2E pingER & perfSONAR (multi NRENs)
GigaMon: capture packets outside border on 10Gbps links and inspect
The upside down pylon is the dashboard representation of the core networks with its redundant links, and the colors showing the current utilization, Clicking on the link brings up the time-series graph to the left. The topology map to the left is autmatically generated and shows the 2 border routers and their connections..
The orange unit is the Gigamon.

13. Security Improved security via ACLs, firewalls,
New VPN infrastructure going into place using IPSEC,
Easy to use visitor network, reasonable security
private VLANs,
blocking of in-bound sessions and outbound SMTP
Blocking of outbound SMTP

14. Future
Developing new roadmap for service types with differing security requirements:
science; business; guest/visitors; SLAC general networks (desktops etc.); internal networks such as controls, data acquisition
being ready to address IPv6 when DoE demands it
Network equipment IPv6 capable
better IP address management with delegation,
Mobile computing and unified communications
Wish: upgrading the wireless network with improved security, reliability, higher speeds, manageability and allow it to be an integral part of our enterprise networking rather than a bolt on overlay. This is possible with the today’s new technologies.
IPv6: have an IPv6 address block from ESnet, will request provider independent block from ARIN, have done some experiments with network application, need to renew, get training, start small with external (www, DNS, )