What we learnt building Carrier Neutral Cloud + Examples Malcolm Siegel - SAFNOG/iWeek 2017

Malcolm Siegel - SAFNOG/iWeek 2017 Introduction Me.. “A wife asked her software engineer husband” "Could you please go shopping for me and buy one carton of milk, and if they have eggs, get 6!" A short time later the husband comes back with 6 cartons of milk. The wife asks him, "Why the hell did you buy 6 cartons of milk?" He replied, "They had eggs." Carrier Neutral Cloud What is it? Why do it? Self regulation is key Open networking Bring your ISP Model Please ask Questions.. Malcolm Siegel - SAFNOG/iWeek 2017

Malcolm Siegel - SAFNOG/iWeek 2017 Our Cloud Using VMWare NSX Dell servers NetApp We Chose VMWARE due to the self service elements. Also very well supported NSX – goes with our Self Service – IE. Customers can configure BGP OSPF etc. direct into the ESGW Dell and NetAPP chosen for Enterprise customers who prefer to know you are “Well Branded” Malcolm Siegel - SAFNOG/iWeek 2017

Malcolm Siegel - SAFNOG/iWeek 2017 Our Development Model.. Discuss briefly model and work way through. Use 3 tier approach Dev / Int(testing) / PROD DEV – should be the playground; test play destroy redo. INT / Test should mirror PROD – testing should be heavily focused here. PROD – should be controlled environment, change controls etc. Malcolm Siegel - SAFNOG/iWeek 2017

Build it (and rebuild it) rules.. Nothing should HIT PROD without been tested in INT first!!! Documentation AS Built Policies Procedures Remember if you have any doubts - trash and restart. We have 30+ years combined experience ie. already made many mistakes 1Gbps is ok for host traffic – but have 10Gbps options available. 2 full months spent building and destroying before 1st Prod released. We had luxury of time.. Malcolm Siegel - SAFNOG/iWeek 2017

Malcolm Siegel - SAFNOG/iWeek 2017 Our “L2” Networking To Discuss: Underlay and overlay. Not here to tell you what you should be using on your network… Many protocols exist for this… Spine – Spine – L3. Leaf Switches run LAG + iBGP Leaf pod to Leaf pod MP-BGP – EVPN ( Come to Cumulus talk and find out more…) Things we looked at but decided not to.. Access – Dist – CORE Flat L2 Spanning tree is asking for trouble – Friends don’t let friends build L2 networks. Do we get involved in Customer networking ?? Loops ?? National was VPLS – moved to VXLAN to standardize. Mention CHEF, PuPPET, Ansible.. Malcolm Siegel - SAFNOG/iWeek 2017

Quick notes on our network 1) Automation is the key. 2) IPSEC is stable and known tech - easy to implement both on customer and edge.. 3) EVPN and (MP)BGP allowed multiple customer addresses to traverse our core. 4) We needed to be able to rewrite VLANS on EGRESS and INGRESS. 5) We had to NOT get involved with our customer networks. Now for some real world examples. Malcolm Siegel - SAFNOG/iWeek 2017

Malcolm Siegel - SAFNOG/iWeek 2017 CLOUD HYBRID CONNECTIVITY Hybrid Cloud.. To be used for Cloud Burst Migrations DR Malcolm Siegel - SAFNOG/iWeek 2017

Malcolm Siegel - SAFNOG/iWeek 2017 ISP BYOIP Bring you own IP (L3) VLAN/s from ISP Routers to ESGW. VXLAN from ESGW to VMs Malcolm Siegel - SAFNOG/iWeek 2017

Malcolm Siegel - SAFNOG/iWeek 2017 Enterprise Mobility Customer creates SSL VPN to ESGW. LDAP Auth is option (AD etc) Granular Firewall Rules for access Malcolm Siegel - SAFNOG/iWeek 2017

Distributed Firewalling NSX DFW PCI-DSS Security Simplified Malcolm Siegel - SAFNOG/iWeek 2017

Stuff you only learn by breaking it. Use the hardware vendor recommended setting (even if it doesn’t feel right). THEY MIGHT NOT SUPPORT YOU IF YOU DEVIATE !! Keep it simple Keep advanced features hidden (unless required) Qualify customers – can they use it ?? Say no often Change Controls do work It is possible to do maintenance and not take customers offline Document from beginning Build a NOC Network.. Hardware recommended issues: RTFM.. It helps  MTU Issues Network Adapter issues with hosted routers – solved with correct network adapters. Server – Server .. Discuss NOC and how it is built – diagram… Malcolm Siegel - SAFNOG/iWeek 2017

Malcolm Siegel - SAFNOG/iWeek 2017 Conclusions Vendor neutrality has key benefits (specifically in cloud) Easy to connect to current setup Less big bang .. Take time building a platform. Learn from mistakes. Allow as much self service as possible. Automated. Choose your customers. Say thank you often Malcolm Siegel - SAFNOG/iWeek 2017