1. Telcordia - June 21, 2004 1 Internet data-plane signaling - revisiting RSVP Henning Schulzrinne Dept. of Computer Science Columbia University hgs@cs.columbia.edu (with Robert Hancock, Hannes Tschofenig, S. van den Bosch, G. Karagiannis, A. McDonald, X. Fu and others)

2. Telcordia - June 21, 20042 Overview Signaling: application vs. data plane Resource control  DiffServ vs. IntServ What’s wrong with RSVP? Components of a general solution NSIS = NTLP (GIMPS) + {NSLP} + Route change detection

3. Telcordia - June 21, 20043 Signaling – the big picture session signaling datapath signaling AS#1 AS#2 off-path NE SIP proxy server off-path signaling on-path signaling data

4. Telcordia - June 21, 20044 Need for data plane state establishment Differentiated treatment of packets  QoS  firewall (loss = 100% vs. loss = 0%) Mapping state  network address translation (NAT) Counting packets  accounting Other state establishment  setting up active network capsules  MPLS paths  pseudo-wire emulation (PWE) – T1 over IP Related: visit subset of data path nodes, but don’t leave state behind  diagnostics  better traceroute  link speeds, load, loss, packet treatment, …

5. Telcordia - June 21, 20045 On-path vs. off-path signaling On-path (path-coupled): visit subset of routers on data path Off-path (path-decoupled): anything else, but presumably roughly along data path  one proposal: one “touch point” for each AS  bandwidth broker  difficult part is resource tracking, not signaling No fundamental differences in protocol  separate out next-hop discovery to allow re-use

6. Telcordia - June 21, 20046 Differentiated packet handling Not just QOS, but also  firewall  network address translation  accounting and measurement filter management traffic filtering traffic shaping, handling & measurement IntServ DiffServ

7. Telcordia - June 21, 20047 DiffServ  IntServ Filter always uses packet characteristic  5-tuple (protocol, source/destination address + port) + global label (TOS)  multiple “flows” can be mapped to one treatment mechanism DiffServIntServin-band identification TOS 5-tuple? 5-tuple mappingfixedsignaledTCP SYN

8. Telcordia - June 21, 20048 The scaling bogeyman Networks routinely handle large-scale per-flow state  firewalls  NATs scaling = cost per flow is constant (or decreasing) flow numbers are modest:  OC-48 can handle 31,875 DS-0 voice calls  Mean call duration = 9 min  60 requests/second  probably about 3 MB of data partially explained by poor initial RSVP explanations  where flow search time ~ O(N) rather than O(1) likely limitations are in AAA, not router signaling It doesn’t scale!

9. Telcordia - June 21, 20049 RSVP characteristics soft-state = state vanishes if not refreshed two-pass signaling = path discovery + reservation receiver-based resource reservation separation of QoS signaling from routing  with some router feedback

10. Telcordia - June 21, 200410 The problem with RSVP Designed for QoS establishment, used mostly for other things (RSVP-TE) Designed for large-scale IP multicast  customer never materialized  adds significant complexity: receiver-based  PATH + RESV  designed for ASM (any-source) rather than SSM (source-specific)  receiver-based motivated by receiver diversity – not very useful in practice Designed in simpler days (1997):  does not work well with mobile nodes (IP mobility or changing IP addresses)  no support for NATs  security mostly bolted on – non-standard mechanisms  single-purpose, with no clear extensibility model  very primitive transport mechanism either refresh or exponential decay (refresh reduction, RFC 2961)

11. Telcordia - June 21, 200411 The cost of multicast for RSVP reservation styles  multiple senders in same group: shared vs. distinct  sender selection: explicit vs. wildcard receiver-oriented  motivated by heterogeneous  can do leaf-initiated join rather than root- initiated but still need periodic PATH to visit new sub-tree three different flow specs  Sender_TSpec, ADSpec, (TSpec, RSpec)  fairly tightly woven into core protocol state merging and management killer reservation (KR-II)  generally, error handling problematic draft-fu-rsvp-multicast-analysis 1020 4060 1020 4060 20 3030 ResvErr!

12. Telcordia - June 21, 200412 IETF NSIS working group chartered in Dec. 2001, after BOF in March 2001 Motivated by Braden’s two-layer model (draft- lindell-waypoint, draft-braden-2level-signal-arch) Active participation from Roke Manor, Siemens, NEC Europe, Nokia, Samsung, Columbia Based partially on CASP protocol designed by Columbia/Siemens group and prototyped at UKy

13. Telcordia - June 21, 200413 NSIS protocol structure client layer does the real work:  reserve resources  open firewall ports  … messaging layer:  establishes and tears down state  negotiates features and capabilities transport layer:  reliable transport NSLP (C) NTLP (GIMPS) transport layer QoS, NAT/FW, … UDP, TCP, SCTP IP router alert GIMPS

14. Telcordia - June 21, 200414 NSIS properties Network friendly  congestion-controlled  re-use of state across applications application-neutral  add more applications later transport neutral  any reliable protocol  initially, TCP and SCTP  also, UDP for initial probing policy neutral  no particular AAA policy or protocol  interaction with COPS, DIAMETER needs work soft state  per-node time-out  explicit removal of state extensible  data format  negotiation

15. Telcordia - June 21, 200415 NSIS properties, cont'd. Topology hiding  not recommended, but possible Light weight  implementation complexity  security associations (re-use)  may not need kernel implementation

16. Telcordia - June 21, 200416 What is GIMPS? Generic signaling transport service  establishes state along path of data  one sender, typically one receiver can be multiple receivers  multicast (not in initial version)  can be used for QoS per-flow or per-class reservation  but not restricted to that avoid restricting users of protocol (and religious arguments):  sender vs. receiver orientation  more or less closely tied to data path initially, router-by-router (path-coupled) later, network (AS) path (path-decoupled)

17. Telcordia - June 21, 200417 NSIS network model – path- coupled NTLP nodes form NTLP chain not every node processes all client protocols:  non-NTLP node: regular router  omnivorous: processes all NTLP messages  selective: bypassed by NTLP messages with unknown client protocols QoS midcom QoS selective omnivorous NTLP chain

18. Telcordia - June 21, 200418 Network model – path-decoupled Also route network-by-network can combine router-by-router with out-of- path messaging AS 1249 AS15465 AS17 Bandwidth broker NAC NTLP data

19. Telcordia - June 21, 200419 GIMPS messages Regular NTLP messages  establish or tear down state  carry client protocol  datagram (“D”) or connection (“C”) mode Hop-by-hop reliability Generated by any node along the chain

20. Telcordia - June 21, 200420 NSIS transport protocol usage Most signaling messages are small and infrequent but:  not all applications  e.g., mobile code for active networks  digital signatures  re-"dialing" when resources are busy Need:  reliability  to avoid long setup delays  flow control  avoid overloading signaling server  congestion control  avoid overloading network  fragmentation of long signaling messages  in-sequence delivery  avoid race conditions  transport-layer security  integrity, privacy This defines standard reliable transport protocols:  TCP  SCTP Avoid re-inventing wheel  see SIP experience

21. Telcordia - June 21, 200421 GIMPS transport protocol usage One transport connection  many NSLP sessions may use multiple TCP/SCTP ports can use TLS for transport-layer security  compared to IPsec, well-exercised key establishment  not quite clear what the principal is re-use of transport   no overhead of TCP and SCTP session establishment  avoid TLS session setup  better timer estimates  SCTP avoids HOL blocking

22. Telcordia - June 21, 200422 Message forwarding Route stateless or state-full:  stateless: record route and retrace  state-full: based on next-hop information in ‘C’ node Destination:  address  look at destination address  address + record  record route  route  based on recorded route  state forward  based on next-hop state  state backward  based on previous-hop state State:  no-op  leave state as is  ADD  add message (and maybe client) state  DEL  delete message state

23. Telcordia - June 21, 200423 Message format No GIMPS distinction between requests and responses  just routed in different directions  client protocol may define requests and responses Common header defines:  destination flag  state flag  session identifier  traffic selector: identify traffic "covered" by this session  message sequence number  response sequence number  message cookie  avoid IP address impersonation  origin address  may not be data source or sink  destination address or scope common headerextensionsclient protocol data

24. Telcordia - June 21, 200424 Message format, cont'd Limit session lifetime Avoid loops  hop counter Mobility:  dead branch removal flag  branch identifier Record route: gathers up addresses of NSIS nodes visited Route: addresses that NSIS message should visit

25. Telcordia - June 21, 200425 Capability negotiation NSIS has named capabilities  including client protocols Three mechanisms:  discovery: count capabilities along a path "10 out of 15 can do QoS"  record: record capabilities for each node  require: for scout message, only stop once node supports all capabilities (or-of-and) avoid protocol versioning

26. Telcordia - June 21, 200426 Next-hop discovery Next-in-path service  enhanced routing protocols  distribute information about node capabilities in OSPF  routing protocol with probing  service discovery, e.g., SLP  first hop, e.g., router advertisements  DHCP  scout protocol Next AS service (not in current version):  touch down once per autonomous system (AS)  new DNS name space: ASN.as.arpa, e.g., 17.as.arpa  use new DNS NAPTR and SRV for lookup similar to SIP approach

27. Telcordia - June 21, 200427 Next-hop discovery scout messages are special NSIS messages limited < MTU size addressed to session destination UDP with router alert option  get looked at by each router reflected when matching NSIS node found next IP hop NSIS-aware? existing transport connection? use D mode to find next NSIS hop establish transport connection N Y N Y done

28. Telcordia - June 21, 200428 Mobility and route changes avoids session identification by end point addresses avoid use of traffic selector as session identifier remove dead branch discovers new route on refresh ADD B=2 DEL (B=2) B=1

29. Telcordia - June 21, 200429 QoS-NSLP: resource reservation NSLP for signaling QoS reservations in the Internet both sender- and receiver-initiated reservations soft-state peer-to-peer signaling and refresh (rather than end-to-end) bundled sessions (e.g., video + audio) agnostic about QoS models (IntServ, DiffServ, RMD, …)

30. Telcordia - June 21, 200430 QoS-NSLP node architecture GIMPS QoS-NSLP resource management policy control packet scheduler packet classifier outgoing interface selection (forwarding) input packet processing traffic control select GIMPS packets API forwarding table manipulation

31. Telcordia - June 21, 200431 QoS-NSLP actors flow sender QNIQNRQNE (…) flow receiver IP address = flow source address QoS unaware NSLP nodes not shown IP address = flow destination data GIMPS+ QoS-NSLP e.g., access router

32. Telcordia - June 21, 200432 QoS-NSLP: sender-initiated reservation RESERVE RESPONSE QNIQNE QNR ( RSN #3) ( RSN #17) ( RSN #4)

33. Telcordia - June 21, 200433 QoS-NSLP: receiver-initiated reservation QUERY RESERVE QNIQNE QNR RESPONSE

34. Telcordia - June 21, 200434 QoS flow aggregation aggregate QoS-NSLP style (RFC 3175) traffic sink (LAN) sinktree style (BGRP)

35. Telcordia - June 21, 200435 The weight of NSIS NSIS state = transport state + GIMPS state + NSLP state GIMPS state = two sockets transport state = O(100) bytes  10,000 users consume 1 MB

36. Telcordia - June 21, 200436 Route change detection Don’t want to wait for periodic rediscovery – delay of 30s+ Not all route changes matter  e.g., only changes between NSIS routers Data plane detection  TTL change of arriving data packets  propagation delay change for data packets  monitoring propagation delay (~ min(e2e delay))  increases in packet loss or jitter

37. Telcordia - June 21, 200437 Route change measurements 12 measurement sites (looking glass) one traceroute every 15’  2.75 hours per pair availability: 99.8% 0.1% repeated IP addresses 4.4% single hop with multiple IP addresses 422 route changes observed after data cleanup (13,074 records) 67 out of 422 also showed AS changes  often, indicates multi-homing

38. Telcordia - June 21, 200438 Route changes

39. Telcordia - June 21, 200439 On-going and planned work Finish NTLP (GIMPS) and NSIS clients (NAT-FW and QoS) Longer term: off-path signaling (new WG?) New applications: diagnostics Mobility support

40. Telcordia - June 21, 200440 Conclusion NSIS = unified infrastructure for data-affiliated sessions avoid making assumptions except that sessions wants to "visit" data nodes or networks not just mobility, but also mobility  route change detection challenging protocol framework in place  but need to work out packet formats