connect communicate collaborate Design and Set Up of the New GÉANT NOC Toby Rodwell, DANTE TNC09, 9 June 2009

connect communicate collaborate Agenda Introduction to DANTE Motivation for an in-house NOC NOC set up “DANTE Dashboard”

connect communicate collaborate Background DANTE established in 1993 Formed by European NRENs Purpose: set up and run a European backbone data network – EuropaNET Initially DANTE only let and managed the contract Later, DANTE designed then operated networks But until 2009 (GN3 project), NOCs always outsourced

connect communicate collaborate Outsourced NOCs Advantages Instant expertise Fixed costs Guaranteed service Economies of scale But inflexible Difficult to walk away from Difficult to re-task

connect communicate collaborate NOC Project Plan Set up … In-house NOC, for network management and control Outsource … Service Desk, for routine tasks and communications Procure … Trouble Ticket System (existing TTS proprietary to GÉANT2 NOC) Late summer 2008, contract awarded to Magdalene Ltd (UK) For Tier 1 NOC (Service Desk) and Trouble Ticket System

connect communicate collaborate Tier Responsibilities Tier 1 First point of contact (new questions, requests, notifications) Detect and report new network incidents Tier 2 Investigate and repair basic network problems Configure and upgrade network elements Tier 3 Advanced trouble shooting and configuration

connect communicate collaborate Network Control Centre Staff NOC Manager Two Tier 3 engineers (IP and optical specialists) Six Tier 2 engineers (two shifts of three) System Administrator Working hours Monday – Friday 0500 – 2100 (UK local), inc Public Holidays (except Christmas and New Year’s Day) On-call engineer out of hours Located in Cambridge, UK (400m from main DANTE office)

connect communicate collaborate

NCC Engineers NCC responsible for both Juniper backbone routers and Alcatel-Lucent transmission and switching equipment Small number of engineers, so all engineers must be competent in both areas Qualities sought Enthusiasm for networking and ability to learn quickly Good theoretical knowledge and a desire to understand concepts NOC or similar experience important but not essential Equipment experience desirable but not expected

connect communicate collaborate

DANTE Dashboard Developed by DANTE staff (in-house and contractor) Required to provide SD with clear, accurate and unambiguous information for new trouble tickets (SD assumed to operate without any knowledge of network topology) Receives SNMP traps from e.g. Juniper routers, Alcatel Network Management System Correlates received alarms and displays (or updates) single alarm on dashboard Integration with DANTE Ops database for meaningful/friendly names

connect communicate collaborate Questions?

connect communicate collaborate

SDE Features (i) Calendar view for Planned Maintenance, to help NOC spot conflicting work

connect communicate collaborate SDE Features (ii) Configuration Item database populated (and updated from) DANTE’s ‘Ops Database’ Help DANTE track trends in poorly performing circuits and hardware Help NOC engineers determine ‘affected services’ given a defective dark fibre, circuit or network element

connect communicate collaborate GÉANT NOC - Requirements 24/7 operation Reliable response to calls and s Prompt alerting of network faults Devolvement of responsibilities Flexibility in re-tasking For GÉANT, EUMEDCONNECT and the End-to-End Coordination Unit (E2ECU)

NOC LOCATION (CAMBRIDGE, UK)

connect communicate collaborate SDE ‘Self Service’ Basic web interface for users to add and view tickets Updating tickets directly not supported (but can be sent to SD from the website)