Steve Peck Service Desk Manager Global Research NOC At Indiana University

Greetings to the MAX Community !!! The Global Research NOC at Indiana University, in conjunction with the staff of the MAX, is here to provide you with excellent SERVICE & SUPPORT!!!

Service Start-Up  This coming Monday June 29, 2009  Final stage of “BETA” support, where the GRNOC assumes full front-line support for the MAX network.  Support aspects are ready, putting the finishing touches on tools, etc.

GRNOC Supported Projects OmniPoP

GRNOC Supported Projects  AMPATH  REN-ISAC  Open Science Grid (OSG)

Service Desk  Performs front-line network management  Provides customer interaction and service  Provides service interface to engineering  Staffed by network technicians  Staff are network knowledgeable; they understand network topology, cause & effect, and management processes  Operates 24 x 7 x 365  Fail-safe operations  Single point of contact for network management  Leveraged resources for cost-effective operations and management

Service Desk  ManagerSteve Peck  SupervisorsNizar Louly Stacy Bengochea Adam Williamson Matt Robertson  Front Line Staff18 technicians  Operations John Wilson & Support Team 3 others

Service Desk  MAX Specialized Support Technician (SST): Terry Broadstreet  MAX NOC Contact Information: Phone: (301) Web form to open a ticket: (coming very soon!!!)

Engineering  GRNOC Network Engineering Team provides Tier 2 support. Receives tickets via round-robin assignment, and provides 24x7 on-call service. Staff of 8, managed by Jay Duncan.  GRNOC Network Planning Team (NPT) provides Tier 3 engineering support.  Jeff Ambern is MAX NPT  Assisted by Tom Johnson  6 other NPT engineers on staff

Engineering  MAX Engineering team provides:  Tier 4 escalation support  Network design  Schedules maintenance activity  New participant & peer connections  Hands & Eyes support

Problem Management Cycle  Alert  Pro-active monitoring  Problem reported by the MAX community  Problem verification & assessment  WHO is affected???  Trouble ticket creation  Member or peer is contacted  Priority of ticket set in accordance with our Customer Impact Guidelines

Problem Management Cycle  Member & Peer Notification Subject: MAX GigaPOP Layer 3 Core Node CLPK Maintenance Issue=127 SUBJECT: MAX GigaPOP Layer 3 Core Node CLPK Maintenance AFFECTED: Layer 3 Core Node CLPK and its associated Peers and Participants START TIME: Tuesday, June 25, 2009, 7:00 AM (0700) UTC (3:00 AM EDT) END TIME: Tuesday, June 25, 2009, 8:00 AM (1000) UTC (6:00 AM EDT) DESCRIPTION: Layer 3 Core Node CLPK will be unavailable while MAX Engineers perform a routing engine swap on the CLPK-T640. A brief outage is expected; however, the entire window is reserved. TICKET NO.: 127:112 TIMESTAMP: :42:25 UTC

Problem Management Cycle  Ticket assignment and escalation  GRNOC Network Engineering  Vendor contact & ticket/case opened  GRNOC Network Planning Team  MAX Engineering  Member and peer notification updates continue until…  PROBLEM RESOLUTION!!!  Customer contact satisfaction

Change Management  Hardware and Software maintenance is scheduled with one week advance notice. Notification will be sent to the affected MAX community accordingly.  Emergency maintenance is performed as needed, for the HEALTH of the network. Notification is sent with as much advance notice as possible.

Customer Impact Guidelines Customer Impact is a subjective scale of measure used to quantify the current impact of a problem or maintenance on the customer’s operations, performance, and usability. Set in conjunction with the customer, it may change as the ticket progresses and is used for ticket escalation. Customer Impact seeks to answer, “How high of a priority is the problem/maintenance to the customer?”

Customer Impact Guidelines 1 - CRITICAL  A problem or issue for which the customer needs immediate, undivided attention from NOC staff until resolved.  The customer is expected to be available immediately to commit full-time resources until the situation is resolved.  The NOC uses this by default when the network is monitored to have an outage of a non-redundant core network element.

Customer Impact Guidelines 2 - HIGH  A problem or issue for which the customer needs resolution within 1 business day.  The customer is expected to commit resources to resolve the situation between the hours of 1300 and 0100 UTC (1200 and 0000 UTC when Daylight Saving Time is in effect).  The NOC uses this by default when the network is monitored to have an outage of a redundant core network element.

Customer Impact Guidelines 3 - ELEVATED  A problem or issue for which the customer does not need immediate resolution, but needs NOC attention within 3 business days.  The customer is expected to be available to provide information or assistance when available during normal business hours.  The NOC uses this by default when a customer connection or session is monitored to have a problem or outage. This is also used by default for maintenance which is both NOC initiated and customer impacting.

Customer Impact Guidelines 4 - NORMAL  No impact to the customer’s operations, performance, and usability  Non-urgent customer service requests  Routine installation/provisioning tickets, non-customer impacting maintenance, and customer initiated maintenance.

Systems Engineering  Provides all the wonderful systems and tools that the NOC uses, as well as tools that are available to the MAX community that are available on the MAX NOC homepage: (web pages coming very soon!!!)  Staff of 12, managed by Luke Fowler

TOOLS: Operations Calendar

TOOLS: Ticket Viewer

TOOLS: Weekly Network Report

TOOLS: SNAPP SMNP Network Analysis and Presentation Package

TOOLS: Real Time Atlas Map

TOOLS: Global NOC Database  The Global NOC Database contains comprehensive information regarding network elements, operations and problem management information for elements, network topology, and contacts.  Drill down linkages to the Database information has been developed for front-end tools such as Footprints and AlertMon.

TOOLS: Global NOC Database

TOOLS: AlertMon The AlertMon front-end to monitoring systems (Nagios) consolidates alerts from network monitoring systems, provides an alert management interface, and links to ticketing and documentation for alerted elements. AlertMon receives input from Nagios and the MAX Database.

TOOLS: AlertMon

TOOLS: Footprints Ticketing

TOOLS: TickMon

Reminders !!!  is being monitored and answered by the MAX NOC Service Desk, not the MAX engineers.  Please respond to our contact information update requests. Very important!  We are still learning, but are 100% committed to doing our best!

Contact the MAX NOC Phone: (301) Web form to open a ticket: (coming very soon!!!) MAX NOC Web Pages (coming very soon!!!)

Questions or Comments??? Steve Peck (317)