GrangeNet Dr. Greg Wickham APAN NOC 25 August 2005

GrangeNet II Background Architecture Services Status Summary

GrangeNet II Background Architecture Services Photos Summary

Background Funded by DCITA - The Advanced Networks Program (ANP) GrangeNet is a non-entity joint venture partnership of The consortium received $14M in 2002 and $4M in 2005 from the Australian Government under the Advanced Networks Program of the Building IT Strengths (BITS) initiative. GrangeNet will run until December 2006

Background Single pair of fibres Brisbane to Melbourne Managed DWDM Backbone OC 48 Circuits 2 Gbit/sec Bandwidth swap to Perth (3456km)

Background “Throwing technology at a community doesn't work. You need people to understand what it will do for them and then they need to change their behavior.“ GrangeNet works with many communities in ICT, the arts, sciences, humanities, education and engineering. GrangeNet facilitates collaboration between communities (layers 1 through 8)

Background WAN had greater capacity than LAN –Forced members to think about their internal networks Subscription based model –Force members to think about their internal charging policies –Network usage (volume no longer an impediment)

GrangeNet II Background Architecture Services Photos Summary

Architecture Grangenet I (A Layer 3 Network) –GSR –OSR 7609 GrangeNet II (A Layer 1,2,3 Network) –Cisco 7609 –ONS E

Architecture A network based on the latest generation of hardware and software Capable of: –Layer 1 – Light Paths –Layer 2 – VLANS –Layer 3 – Routing Carrying: –IPV4 Unicast and Multicast –IPV6 Unicast

Architecture

Hardware Summary Layer 3 and Layer 2 –Cisco 7609 –3BXL Supervisor –3BXL DFC –67xx series line cards Layer 1 –ONS 16454E –OC48 backbone –Gigabit Interconnects

Hardware – Cisco Gbps switching capacity (40 Gbps / slot) 1GB of DRAM Sup720-3BXL IPv4 routing to 400Mpps; and IPv6 to 200Mpps Routes 1,000,000 (IPv4); 500,000 (IPv6) MPLS in hardware L2 bridging up to 400Mpps All line cards are fabric enabled with 3BXL distriubuted forwarding cards (except FlexWan)

Hardware – ONS 15454E Hardware: –Cisco ONS 15454E Capability: –Time Division Multiplexing –OC48 –Gigabit –Expandable to 10 Gigabit –Carrier Class Reliability

Hardware – 15454E Some assembly required ….

Architecture

ONS 15454E aggregates Gigabit into OC48 circuits

Architecture Pairs of OC48 circuits carried on managed DWDM system

Architecture Etherchannel Trunks on inter-capital links

Architecture

The 15454Es are used to Time Division Multiplex multiple Gigabit Circuits onto an OC48 Gigabit circuits between 7609s are port channelled Port channel groups between routers are 802.1q trunks

Architecture - Redundancy All GrangeNet routers are fully redundant –Dual power supplies –Dual processors (Supervisor or TCC2) –Dual line cards However –Backbone carried over a Single fibre path But –Very few fibre breaks –More scheduled outages than unscheduled

GrangeNet II Background Architecture Services Photos Summary

GrangeNet II Services With the implementation of GrangeNet II more services can be offered than just the traditional R&E (routed protocols) The service offerings are: –GrangeNet R&E - Layer 3 (Network Layer) –GrangeNet LAN - Layer 2 (Data link layer) –GrangeNet Lightpath - Layer 1 (Physical Layer)

Provisioning

Cisco 7609 handles all layer 3 (IP routed) and layer 2 (Vlan) switching

Provisioning Cisco ONS 15454E handles all layer 1 (TDM) switching

GrangeNet R&E GrangeNet R&E is the traditional layer 3 service that permits members to peer with both local and international Research and Education members. IPv4 –Unicast –Multicast IPv6 –Unicast –Multicast (2006*)

GrangeNet LAN An complementary service to MPLS is GrangeNet LAN - a Layer 2 service. This is a similar offering to GrangeNet Lightpath however the members traffic is carried across the GrangeNet backbone in dedicated Vlans. There are currently no capacity or quantity constraints on the LAN service offering. A client can request many LAN services and have these combined with a R&E service onto the same physical GrangeNet connection.

GrangeNet LAN Negotiations: –VLAN number – Must be unique between all sites and GrangeNet(*) –Path (VLANs are statically mapped to bearers) –Tagging (untagged or tagged) Any VLAN can be –extended to any POP site –extended to ALL POP sites GrangeNet is a big ‘switch’

GrangeNet LAN GrangeNet Member can send untagged traffic and it will be tagged and carried through the network

GrangeNet GrangeNet LAN Members can send multiple VLANs down the same physical circuit Tag 500 Tag 501 Tag 493

GrangeNet GrangeNet LAN Campus A Campus B Campus C Tag 501Tag 500Tag 493

GrangeNet LAN Shared bandwidth with all other subscribers Member security is an issue

GrangeNet LAN Management Issues: –Not possible to determine bandwidth usage per VLAN –Configuration implicitly trusts Cisco more than Members –Members faced with security issues Why not MPLS?

GrangeNet Lightpath GrangeNet Lightpath is a Layer 1 clear channel service that transports data between member endpoints. Satisfies members security constraints. The implementation of the Lightpath service is at a TDM (Time Division Multiplexing) level direct onto the GrangeNet DWDM backbone.

Provisioning - L1 / LightPath Reserved bandwidth Minimal security implications

Architecture

Provisioning - L1 / LightPath GrangeNet The Physical The Reality Bringing researchers together …

Using a Light Path Provisioning: –Currently a manual process Uses Cisco Transport Controller Takes minutes –UCLP (From CaNARIE) (in deployment) Connectivity: –User needs a connection to two GrangeNet POPs

GrangeNet II Background Architecture Services Photos Summary

Photos

GrangeNet II Background Architecture Services Photos Summary

Layer 3 – Domestic and International R&E Layer 2 – VLAN –Replaced use of MPLS –Challenges users to think about the way they use networks Layer 1 – Lightpath –Last mile limitation

Summary On the plus side –7609 is an excellent routing platform –15454E very simple to use –A very stable and a mostly manageable solution –Flat network is extremely flexible

Questions?