Internet Indirection Infrastructure Ion Stoica UC Berkeley June 10, 2002

Collaborators Students: –Daniel Adkins –Karthik Lakshminarayanan –Ananth Rajagopala-Rao –Sonesh Surana –Shelley Zhuang Postdocs: –Kevin Lai Faculty: –Randy Katz –Scott Shenker

Motivations Today’s Internet is built around a point-to- point communication abstraction: –Send packet “p” from host “A” to host “B” This abstraction allows Internet to be highly scalable and efficient, but… … not appropriate for applications that require other communications primitives: –Multicast –Anycast –Mobility –…

Why? Point-to-point communication abstraction implicitly assumes that there is one sender and one receiver, and that they are placed at fixed and well-known locations –E.g., a host identified by the IP address xxx.xxx is located in Berkeley

Existing Solutions Change IP to support new services, e.g., –Mobile IP –IP multicast Disadvantages: –Difficult to implement while maintaining Internet’s scalability (e.g., multicast) –Even if they are implemented, ISPs might not have incentive to enable them

Existing Solutions (cont’d) Implement the required functionality at the application level, e.g., –Application level multicast (Narada, Overcast, Scattercast, Fastforward…) –Application level mobility Disadvantages: –Efficiency hard to achieve –Each application implements the same functionality over and over again –No synergy; each application implements usually only one service e.g., services hard to combine

Key Observation All previous solutions use a simple but powerful technique: indirection –Assume a logical or physical indirection point interposed between sender(s) and receiver(s) Examples: –IP multicast assumes a logical indirection point: the IP multicast address –Mobile IP assumes a physical indirection point: the home agent

Our Solution Add an efficient indirection layer (IL) on top of IP –Transparent for legacy applications Use an overlay network to implement IL –Incrementally deployable; don’t need to change IP IP TCP/UDP Application IL

Internet Indirection Infrastructure (i3) Change communication abstraction: instead of point-to-point, exchange data by name –Each packet is associated an identifier ID –To receive a packet with identifier ID, receiver R maintains a trigger (ID, R) into the overlay network SenderReceiver (R) IDR trigger send(ID, data) send(R, data)

Service Model API –sendPacket( p ); –insertTrigger( t ); –removeTrigger( t ); // optional Best-effort service model (like IP) Triggers are periodically refreshed by end- hosts Reliability, congestion control, and flow- control implemented at end-hosts

Discussion A trigger is similar to an entry in a routing table Unlike IP, IL gives the endhosts the ability to control triggers, i.e., endhosts are responsible for setting and maintaining “routing tables”

The Promise Provide support for –Mobility –Multicast –Anycast –Composable services

Mobility Host just needs to update its trigger as it moves from one subnet to another –Robust –Support simultaneous mobility –Can alleviate the “triangle routing problem” –Location privacy Sender Receiver (R1) IDR1 send(ID,data) send(R1, data)

Mobility Host just needs to update its trigger as moves from one subnet to another –Robust –Support simultaneous mobility –Can alleviate the “triangle routing problem” –Location privacy Sender Receiver (R2) IDR2 send(ID,data) send(R2, data)

Multicast Unifies multicast and unicast abstraction –Multicast: receivers insert triggers with the same identifier An application can dynamically switch between multicast and unicast Sender Receiver (R1)IDR1 send(ID,data) send(R1, data) Receiver (R2) IDR2 send(R2, data)

Anycast Generalize the matching scheme used to forward a packet –Until now we assumed exact matching Next, we assume: –Longest prefix matching (LPM) using a prefix larger than a predefined constant l to avoid collisions In the current implementation: ID length, m = 256, l = 128

Anycast (cont’d) Anycast is simply a byproduct of the new matching scheme, e.g., –Each receiver i in the anycast group inserts IDs with the same prefix p and a different suffix s i Sender Receiver (R1) p|s 1 R1 send(p|a,data) Receiver (R2) p|s 2 R2 p|s 3 R3 Receiver (R3) send(R1,data)

Anycast (cont’d) Highly flexible: the least significant l-m bits of ID are application specific Two examples: –Load balancing: clients and servers choose random suffixes –Proximity: encode host locations in their identifiers LPM can also implement “exact”, “range”, >, =, and <= matching rules

Range Matching Example Match any identifier in the range [36, 71] –Use a set of triggers to emulate an arbitrary range …01001xx …0101xxx …011xxxx …1000xxx [36,71] R R R R [32,35] [36,39] [40,47] [48,71]

Composable Services Use a stack of IDs to encode the successions of operations to be performed on data (e.g., transcoding) Advantages –Don’t need to configure path –Load balancing and robustness easy to achieve Sender (MPEG) Receiver R (JPEG) ID_ MPEG/JPEG S_ MPEG/JPEG ID R send((ID_ MPEG/JPEG,ID), data) S_ MPEG/JPEG send(ID, data) send(R, data)

Composable Services (cont’d) Both receivers and senders can specify the operations to be performed on data Receiver R (JPEG) ID_ MPEG/JPEG S_ MPEG/JPEG ID (ID_ MPEG/JPEG, R) send(ID, data) S_ MPEG/JPEG Sender (MPEG) send((ID _MPEG/JPEG, R), data) send(R, data)

Example: Heterogeneous Multicast Sender not aware of the transformations Receiver R1 (JPEG) ID_ MPEG/JPEG S_ MPEG/JPEG ID (ID_ MPEG/JPEG, R1) send(ID, data) S_ MPEG/JPEG Sender (MPEG) send((ID _MPEG/JPEG, R1), data) send(R1, data) ID R2 Receiver R2 (MPEG) send(R2, data)

Discussion: i3 vs. IP Multicast i3 doesn’t provide direct support for scalable multicast –Simple to implement Give end-hosts the ability to control “routing” –End-hosts can build their own multicast tree if needed! R2R2 R1R1 R4R4 R3R3 g R 2 g R 1 gxgx x R 4 x R 3 (g, data)

Implementation i3 is implemented on top of Chord –But can easily use CAN, Pastry, or Tapestry Each trigger ( id, …) is stored on the server responsible for id Use Chord routing to find best matching trigger for a packet ( id, data )

Implementation: Chord Define a circular ID space 0..2 m-1 Each server has a unique identifier in the ID space An arbitrary ID id is mapped on the server whose ID follows id A trigger ( id, …) is stored on the server mapping id Need O(log N) hops to locate the servers for an arbitrary ID –N, number of servers in the system R m = 6 i3 server

Implementation At minimum, an end-host needs to know only one i3 server, e.g., –S knows server 3, R knows server R R S R trigger(37,R) send(37, data) send(R, data)

Optimizations Sender/receiver caches server that maps a specific ID –Subsequent packets are sent via one i3 server R S R send(R, data) send(37, data) cache server “41”

Optimizations (cont’d) Alleviate triangular routing problem –Pick identifiers that maps on servers nearby by sampling identifier space (can be done off-line) Use public triggers for initial randezvous Use private triggers for efficient routing

Optimizations (cont’d) H1 wants to contact H2 –H1 knows H2’s public trigger (e.g., Hash(cnn.com)) id public H2 H1 id1 private id2 private H2 H1 H2 send(id public, id1 private ) send(H2, id1 private ) send(id1 private, id2 private ) send(id2 private, data) send(H2, data)

Status Prototype deployed on –Millennium: ran over > 200 i3 servers –Small Internet testbed (13 end-hosts in US, Europe and Asia) i3 header with one identifier  48 bytes Preliminary results (on Millennium): –PIII, 700 MHz, running Linux –Trigger insertion: 80,000 triggers/sec –Throughput of data forwarding overhead 35,500 pps (0 byte payload) 23,300 pps (1,400 byte payload); 261 Mbps

Applications: Support for Mobility Robustness –Indirection point maintained by infrastructure –Use multicast for hand-over between cells Efficiency –Latency stretch ~ 1.4 Support simultaneous end-host mobility Support for legacy applications

Application: Reliable and Scalable Multicast End-hosts responsible for tree construction and recovery Robustness –Multicast tree maintained by infrastructure Efficiency –Use anycast and limited-scope multicast to implement recovery and construct the tree

Conclusions Indirection – key technique to implement basic communication abstractions and services such as –Multicast, Anycast, Mobility, … This research advocates for: –Leaving IP do what is doing best: point-to-point unicast communication – Building an efficient Indirection Layer on top of IP General research theme: explore new communication abstractions for overlay network based infrastructures –This work proposes a rendezvous-based communication abstraction –Other abstractions?

Future Work Security –Denial of service attacks –Trigger hijacking “Completeness” of communication abstraction –Matching rules for anycast –Composable services

Backup Slides

Load Balancing Each server chooses the last l-m bits of its ID randomly Each client sets the last l-m bits of packets’ IDs randomly 1010:0010 S1 1010:0101 S2 1010:1010 S3 1010:1101 S4 S1 S2 S3 S4 A B send(1010:0110,data) send(1010:1110,data)

Proximity The last l-m bits of the servers’ and packets’ have location semantics 1000:0010 S1 1000:1010 S2 1000:1101 S3 S1 S2 S3 send(1000:0110,data)