Fundamentals of Network Design JIM LOCKARD, PGMP, PMP, ENP

Statistics Approximately 30 States have begun created plans or Concept of Operations documents for the design and implementation of NG9-1-1 capabilities Over 25 have begun deployment of ESInet’s to support NG9-1-1 Plus many more on the local / regional level

What Is an ESInet? Per as defined: ◦ An ESInet is a managed IP network that is used for emergency services communications, and which can be shared by all public safety agencies. It provides the IP transport infrastructure upon which independent application platforms and core functional processes can be deployed, including, but not restricted to, those necessary for providing NG9-1-1 services. ESInets may be constructed from a mix of dedicated and shared facilities. ESInets may be interconnected at local, regional, state, federal, national and international levels to form an IP-based inter-network (network of networks). Do not assume an isolated network (walled garden)

What Is ESIND? ESInet Design (ESIND) is the working group responsible for producing an informational document on designing an Emergency Services IP Network (ESInet) The current NENA document (Emergency Services IP Network Design for NG9-1-1) released December 14, 2011 New version finishing review NENA-INF X

ESIND overview Covers the design of ESInets at OSI layers 1, 2, and 3 Planning considerations, Design suggestions and implementation methodologies are discussed Performance requirements ◦ Service level agreements for operators ◦ Aspects of network security ◦ Prioritization ◦ Traffic engineering ◦ Network management and monitoring

Informational document, not normative Target audience includes agencies responsible for procuring an ESInet and network architects tasked with designing it Provides an overview of the technologies available to build an ESInet and guidance on reliability and performance Does not replace proper training ESIND should be used IN CONJUNCTION with other standards to ensure linkage

Work group effort New ESIND work group opened to specifically address: ◦ Autonomous System’s ◦ DNS ◦ New service models ◦ Refine metrics ◦ Review entire document

New ESIND considerations How to use wireless links ◦ Microwave ◦ Can FirstNET impact me ◦ Can I use WiFi ◦ What about Satellite Autonomous System Other Internet Access at a PSAP Multiple service providers How to gauge / assess diversity and redundancy

What can make up an ESInet Essentially an IP network The Core services are: ◦ IP routing ◦ Security ◦ Switching ◦ Gateways ◦ Interconnections  Local connections  Telco connections  Outside agency connections

ESInet and i3 ESInet creates the capability for the functions to operate i3 functions utilize the ESInet for transport, delivery, end to end connectivity ◦ BCF ◦ ESRP ◦ ECRF ◦ LNG ◦ LPG ◦ LSRG

Emergency Services IP network (ESInet) NOT a walled garden ◦ All ESInet elements assume ESInet is the open Internet The ESInet IS connected to the Internet All public safety, not just Bottom up design – local ESInets interconnected to form state ESInet (with optional state backbone), state ESInets interconnected to form national ESInet PSTNPSTN INTERNETINTERNET International National Regional State/Province Regional (county etc.) PSAP Protected Layer

Design considerations Connections Bandwidth Security Service level IP service (MPLS or other) Network Monitoring Performance requirements

Additional considerations Interconnections QoS Traffic Engineering Diversity Redundancy Fault Tolerance

Additional considerations Availability Reliability DNS operation DDoS mitigation Service Level Agreement (SLA) Access guidelines and rules Capabilities

Potential problem areas Providers unfamiliar with ESInet stds ◦ Common enterprise standards can be different than expected ESInet standards ESInets not fully implemented ◦ Networks that are partially implemented as IP networks Specific rules interfering with full implementation ◦ Internal policies that conflict with an ESInet implementation ESInet “edge” integration ◦ ESInets not utilizing common interfaces, protocols etc Transition and Migration ◦ Testing and acceptance challenges

Potential Pitfalls Support staff training ◦ Planning, Scheduling, System soak Pre-cut testing ◦ Misroutes ◦ Switch provisioning Phased system cutover ◦ Delays due to actual emergencies Post-cut testing ◦ Unresolved issues – non-production impacts (encryption etc) System acceptance

Layer 1: Physical This layer deals with cabling and communication medium Typical circuits to connect to an ESInet: ◦ Copper (ex. T1, T3) ◦ Optical Fiber ◦ Radio and Satellite (ex. 3G/4G, microwave) Most important to consider is path diversity ◦ Get connectivity from different sources ◦ Circuits should follow different routes

Layer 2: Data Link Responsible for framing traffic on the wire Highly dependant on physical layer Availability depends on local service providers Examples: ◦ HDLC (T1/T3): multiple circuits can be bonded to increase bandwidth ◦ Frame Relay: not recommended

Layer 2: Data Link (cont.) ATM: supports different classes of services: o CBR: Constant bandwidth, more expensive o VBR: Variable bandwidth, less expensive o UBR: Unspecified bandwidth, least expensive Metro Ethernet: o Ethernet over a wide area o Wide variety of providers MPLS: o Tends to replace ATM and Frame Relay

Layer 3: Network Two addressing schemes: ◦ IPv4:  Widely deployed (used by 98% of devices)  Public addresses are soon to be exhausted  There are mechanisms to lessen effect of exhaustion such as private IP addresses and NAT ◦ IPv6:  Practically endless supply of addresses  Interoperability with IPv4 difficult  Not widely implemented by software vendors ◦ Best practice to design ESInet with both IPv4 and IPv6 (dual stack)

Layer 3: Network (cont.) Dynamic routing protocols: ◦ Used to discover network topology and determining route availability ◦ Best practice to utilize a routing protocol within an ESInet ◦ Autonomous System:  An AS is a network or group of networks under a single administration  Best practice to configure regional ESInets as their own AS

Layer 3: Network (cont.) ◦ Interior Gateway Protocols:  Used within an AS  OSPF: ◦ Widely deployed ◦ Version 2 is for IPv4 ◦ Version 3 is for IPv6 (recommended)  EIGRP (Enhanced Interior Gateway Protocol): ◦ Proprietary to Cisco  IS-IS (Intermediate System): ◦ Commonly used on large deployments ◦ BGP- Border Gateway Protocol:  Provide routing between AS  Protocol of choice when dealing with untrusted networks  Should be used at the state and national level

Traffic Engineering ESInet should provide non-blocking service to high priority traffic Network design must take into account not only current requirements but future as well, such as video ◦ If it is not financially feasible to build the ESInet with capacity that will only be used in the future, then it should be easy to add capacity when time comes When designing for video, a possible formula for determining bandwidth is 2Mbps per PSAP + 2Mbps per position that will take video

Traffic Engineering (cont.) When interconnecting an ESInet with the Internet, care should be taken to mitigate the impact of a Distributed Denial of Service (DDoS) attack and still provide service Service providers may monitor incoming traffic from an ESInet and may discard any traffic that exceeds the rate purchased ◦ Traffic shaping should be applied before transmission to the service provider so it meets the provider’s acceptable rate

Traffic Engineering (cont.) In legacy telephony, deployed capacity cannot be exceeded ◦ Once capacity is used up, any new call gets a busy On the other hand, an ESInet will happily accept a new call when capacity is exceeded ◦ In which case all calls will be affected as packets will be dropped randomly ◦ The ESRP should be used to perform Call Admission Control to prevent capacity to be exceeded

Traffic Engineering (cont.) Quality of Service (QoS) provides different priorities to different data flows ◦ Only useful if implemented end-to-end ◦ Service provider’s ability to support QoS should be taken into account ◦ ESInets utilizes DSCP to implement QoS ◦ NG9-1-1 Functional Elements must mark their traffic appropriately

Hardware/Network Elements Elements used to build an ESInet should: ◦ Be highly reliable ◦ Have a proven track record ◦ Have a warranty ◦ Have qualified personnel to support it ◦ Have a vendor that provides 24/7 support ◦ Have an acceptable MTTR ◦ Be scalable

Network Security Applicable NENA standards: ◦ Security for Next-Generation Standard (NG-SEC) contains a number of sections applicable to an ESInet ◦ Detailed Functional and Interface Specification for the NENA i3 Solution – Stage 3 also contains requirements including encryption and authentication Border Control Functions should be deployed upstream and downstream from an ESInet ◦ Logs and alerts should be monitored

Network Management and Monitoring An ESInet should be monitored ◦ Not a regulation requirement but may well be ◦ All circuits and network elements should be monitored ◦ All network components should generate SNMP traps to a network management system ◦ Non-network components such as NG9-1-1 functional elements should also utilize SNMP traps

Network Management and Monitoring (cont.) Proper network management requires: ◦ Accurate and up-to-date documentation ◦ Demarcation points ◦ Contacts and escalation lists ◦ SLA benchmarks ◦ Capacity management/trending ◦ Monitoring the state of element configuration (ex. QoS) ◦ Configuration management / change control

Performance Requirements Packet loss, jitter and delay all affect the quality of multimedia traffic Packet loss: ◦ Over 5% affects intelligibility ◦ Should be kept below 2.5% and 1% is achievable Jitter: ◦ Describes the variable delay packets experience crossing the network ◦ Jitter buffers are used in endpoints to smooth out variation ◦ Jitter should be bounded to less then 20 ms

Performance Requirements (cont.) Latency: ◦ Time required for a packet to reach its destination ◦ When mouth to ear latency exceeds 150 ms, full duplex conversation becomes difficult ◦ Best to keep latency in ESInet less than 15 to 20 ms

Service Level Agreement It is a contractual agreement with a vendor on a service level commitment ◦ Typically represented as uptime (ex. 99.9%, 99.99%, %) ◦ Typically describes where, how and how often measurements are made ◦ Typically uses impact levels to define severity of outage ◦ May include financial penalties if agreement is not met ◦ Should also include response times for repairs and if on-site spares are required

Service Level Agreement (cont.) ATM, Metro Ethernet and MPLS vendors usually share their network capacity among multiple customers ◦ Unless an SLA is in place, there is no guarantee that ESInet traffic will have preferential treatment Redundant systems should be regularly verified with deliberate fail-over tests

ESIND – i3 links ESInet’s ◦ Physical connections - capabilities ◦ Routers / Switches ◦ Interconnections ◦ Redundant networks ◦ Bandwidth ◦ The foundation for applications and functions NG9-1-1 Functions o Logical internetwork - functions o Functional elements o Internetworking - Network addressing (IP, SIP, etc) o Security o Applications o Utilization of the ESInet

ESIND overview Meant to be used in conjunction with Standards documents, not as a substitute for them A summary of the core requirements for an ESInet are included in the NENA v 2.0 Detailed Functional and Interface Specification for the NENA i3 Solution – Stage 3 are as follows: ◦ The network between the PSAP and the ESInet will be a private or virtual private network based upon TCP/IP ◦ It will have scalable bandwidth to support new enhanced services. ◦ The Emergency Services IP Network shall be a conventional routed IP network ◦ MPLS or other sub-IP mechanisms are permitted as appropriate

ESIND overview (cont) ◦ The PSAP should use redundant local area networks for reliability ◦ PSAP LAN to the ESInet must be resilient, secure, physically diverse, and logically separate ◦ The ESInet shall be engineered to sustain real time traffic, including data, audio, and video ◦ Connections between the PSAP and ESInet shall be secured TCP/IP connections ◦ ESInets should be capable of operating on IPv4 and IPv6 network infrastructures

Thank you!