1. December 2007 Neil Francis Network Administrator SWERN University of Bath 50 VLANS and 3 Telcos

2. 2 Outline

3. 3 Some Background SWERN is in the South West of England Merger of SWAN and BWEMAN Essentially 2 networks connected together Piecemeal build – varied active equipment Some resilience (always a worry) Mixture of ATM, POS, Ethernet etc Limited HR Two part time engineers, network managers, some Ops support etc Geographical issues – long and thin, rural(ish) 13 POPs – some remote The usual stuff.....

5. 5 SWERN 2 For SWERN 2 we had an opportunity to tidy things up The usual design factors Homogeneous, Resilient, Cost Effective?, Engineering, HR. The South West is long and thin We had POPs in the large conurbations Cheltenham Swindon Bath Bristol Taunton Exeter Plymouth

7. 7 SWERN 2 Options Dark fibre, managed solutions etc All considered but not only cost ruled these out. Long distance ethernet circuits becoming more comfortable for us Telcos had solutions available We already had some metro ethernet ccts So Ethernet everywhere... Standard kit (for us) Standard circuit presentations (fibre and copper) Layer 2 network – easier to manage? Keep it simple by using point to point vlans It looked good

8. 8 SWERN 2 - Ethernet Connecting these all together Do loops Two POPs in each city Connect these together Fairly close together so relatively easy Connect each POP back to an RNEP Slightly harder due to distances Connect the RNEPs together Fairly easy End up with something like this....

9. 9 SWERN 2 - Ethernet Essentially 6 loops here Connect to standard ethernet equipment...

10. 10 SWERN 2 – Simple Kit Cisco 7609 OSR Routers Sup. Engine 720/1Gbyte RAM Located at RNEPs Cisco 7206VXR Routers NPE-G1 Processors Cisco Catalyst 4948 48 Port 10/100/1000 Ethernet Switch

11. 11 SWERN 2 – Loops & VLANS SJ5 RNEP s POP s End Sites X X X

12. 12 SWERN2 - Vlans

13. 13 SWERN2 - Tender Tender issued late 2005 Solutions based on mixture of WDM SDH/SONET/EPL LES/WES MPLS But all presented to us as ethernet Carrier Ethernet Metro Ethernet Evolved Ethernet National Ethernet whatever......

14. 14 SWERN2 - Bandwidths All bandwidths uncontended

15. 15 SWERN2 - VLANS With 6 loops 8 VLANS per loop Plus some test VLANS Over 50 VLANS – all terminate on our RNEP routers All point to point VLANS Although we also have a management VLAN Traffic routed via layer 3 We still have to manage, monitor, administer, troubleshoot etc.

16. 16 SWERN 2 - Routing

17. 17 SWERN 2 - Routing OSPF Single small backbone (Area 0) Stub areas from the POP areas Timers tuned low Also looked at BFD for fast link failure detection Best case scenario – as good as POS! Worked between our POP routers and RNEPs Needed 12.4.x IOS on our 7200s Experimented with timers but were finding even a flash disk insertion was causing BFD to baulk We currently rely on fast converging OSPF

18. 18 SWERN2 - Observations We know we have a mixture of the following: Simple LES/WES circuits DWDM/SDH/EPL circuits MPLS circuits We run point to point VLANS across all this technology and it works Not to say everything is perfect

19. 19 SWERN2 – EPL Latency

20. 20 SWERN2 – MPLS Latency

21. 21 SWERN2 - Observations We have a management VLAN Spanning tree should be turned on Have had more than one occurrence of looping Somewhere - BPDUs are not being passed Confidence lost Witnessed CDP neighbor inconsistencies Tracked to provider switches being at different OS levels. A bug here but related to above. Loss of VLANS across the.ad/.ah bridges 3 VLANS across 3 loops down Provider insistent all was working Fixed at 3 rd level support

22. 22 SWERN2 - Observations Management Vlan BFD Resilience OSPF tuned low Do without BFD 50ms Switching Providers not monitoring individual vlans