Internet Indirection Infrastructure Ion Stoica UC Berkeley Nov 14, 2005

2 Motivations Today’s Internet is built around a unicast point-to-point communication abstraction: –Send packet “p” from host “A” to host “B” This abstraction allows Internet to be highly scalable and efficient, but… … not appropriate for applications that require other communications primitives: –Multicast –Anycast –Mobility –…–…

3 Why? Point-to-point communication  implicitly assumes there is one sender and one receiver, and that they are placed at fixed and well-known locations –E.g., a host identified by the IP address xxx.xxx is located in Berkeley

4 IP Solutions Extend IP to support new communication primitives, e.g., –Mobile IP –IP multicast –IP anycast Disadvantages: –Difficult to implement while maintaining Internet’s scalability (e.g., multicast) –Require community wide consensus -- hard to achieve in practice

5 Application Level Solutions Implement the required functionality at the application level, e.g., –Application level multicast (e.g., Narada, Overcast, Scattercast…) –Application level mobility Disadvantages: –Efficiency hard to achieve –Redundancy: each application implements the same functionality over and over again –No synergy: each application implements usually only one service; services hard to combine

6 Key Observation Virtually all previous proposals use indirection, e.g., –Physical indirection point  mobile IP –Logical indirection point  IP multicast “Any problem in computer science can be solved by adding a layer of indirection”

7 Our Solution Use an overlay network to implement this layer –Incrementally deployable; don’t need to change IP Build an efficient indirection layer on top of IP IP TCP/UDP Application Indir. layer

8 Internet Indirection Infrastructure (i3) Each packet is associated an identifier id To receive a packet with identifier id, receiver R maintains a trigger ( id, R) into the overlay network Sender idR trigger iddata Receiver (R) iddata R

9 Service Model API –sendPacket( p ); –insertTrigger( t ); –removeTrigger( t ) // optional Best-effort service model (like IP) Triggers periodically refreshed by end-hosts ID length: 256 bits

10 Mobility Host just needs to update its trigger as it moves from one subnet to another Sender Receiver (R1) Receiver (R2) idR1 idR2

11 iddata Multicast Receivers insert triggers with same identifier Can dynamically switch between multicast and unicast Receiver (R1) idR1 Receiver (R2) idR2 Sender R1data R2data iddata

12 Anycast Use longest prefix matching instead of exact matching –Prefix p: anycast group identifier –Suffix s i : encode application semantics, e.g., location Sender Receiver (R1) p|s 1 R1 Receiver (R2) p|s 2 R2 p|s 3 R3 Receiver (R3) R1 data p|a data p|a data

13 Service Composition: Sender Initiated Use a stack of IDs to encode sequence of operations to be performed on data path Advantages –Don’t need to configure path –Load balancing and robustness easy to achieve Sender Receiver (R) id T T id R Transcoder (T) T,id data iddata R id T,id data id T,id data

14 Service Composition: Receiver Initiated Receiver can also specify the operations to be performed on data Receiver (R) id id F,R Firewall (F) Sender id F F id F,R data R F,R data id data id data

15 Outline  Implementation Examples Security Applications Legacy application support

16 Quick Implementation Overview i3 is implemented on top of Chord –But can easily use CAN, Pastry, Tapestry, etc Each trigger t = ( id, R ) is stored on the node responsible for id Use Chord recursive routing to find best matching trigger for packet p = ( id, data )

17 Routing Example R inserts trigger t = (37, R) ; S sends packet p = (37, data) An end-host needs to know only one i3 node to use i3 –E.g., S knows node 3, R knows node R R S R trigger(37,R) send(37, data) send(R, data) Chord circle S R 0 2 m-1 [8..20] [4..7] [21..35] [36..41] [40..3]

18 Sender (S) Optimization #1: Path Length Sender/receiver caches i3 node mapping a specific ID Subsequent packets are sent via one i3 node [42..3] [4..7] [8..20] [21..35] [36..41] 37R data R cache node Receiver (R)

19 Optimization #2: Triangular Routing Use well-known trigger for initial rendezvous Exchange a pair of (private) triggers well-located Use private triggers to send data traffic [42..3] [4..7] [8..20] [21..35] [36..41] 37R R [2] 2S 37 [2] 2 [30] 30R S [30] 30 data R Receiver (R) Sender (S)

20 Outline Implementation  Examples –Heterogeneous multicast –Scalable Multicast –Load balancing –Proximity Security Applications Legacy application support

21 Example 1: Heterogeneous Multicast Sender not aware of transformations Receiver R1 (JPEG) id_ MPEG/JPEG S_ MPEG/JPEG id (id_ MPEG/JPEG, R1) send(id, data) S_ MPEG/JPEG Sender (MPEG) send((id _MPEG/JPEG, R1), data) send(R1, data) id R2 Receiver R2 (MPEG) send(R2, data)

22 Example 2: Scalable Multicast i3 doesn’t provide direct support for scalable multicast –Triggers with same identifier are mapped onto the same i3 node Solution: have end-hosts build an hierarchy of trigger of bounded degree R2R2 R1R1 R4R4 R3R3 g R 2 g R 1 gxgx x R 4 x R 3 (g, data) (x, data)

23 Example 2: Scalable Multicast (Discussion) Unlike IP multicast, i3 1.Implement only small scale replication  allow infrastructure to remain simple, robust, and scalable 2.Gives end-hosts control on routing  enable end- hosts to –Achieve scalability, and –Optimize tree construction to match their needs, e.g., delay, bandwidth

24 Example 3: Load Balancing Servers insert triggers with IDs that have random suffixes Clients send packets with IDs that have random suffixes S S S S4 S1 S2 S3 S4 A B send( ,data) send( ,data)

25 Example 4: Proximity Suffixes of trigger and packet IDs encode the server and client locations S S S3 S1 S2 S3 send( ,data)

26 Outline Implementation Examples  Security Applications Legacy applications support

27 Some Attacks S R idR Attacker (A) idA Eavesdropping Attacker id 2 id 3 id 1 id 2 id 4 id 3 id 1 id 4 Loop Victim (V) id 3 V Attacker id 2 id 1 id 3 Confluence Attacker id 2 id 1 id 3 id 2 Dead-End

28 Constrained Triggers h l (), h r (): well-known one-way hash functions Use h l (), h r () to constrain trigger (x, y) prefix key must match ID: suffix xy x.key = h l (y) xy x.key = h l (y.key) end-host address Left constrained xy y.key = h r (x) Right constrained

29 Attacks & Defenses Trigger constraints PushbackTrigger challenges Public ID constraints Eavesdropping& Impersonation Loops & Confluences Dead-ends Reflection & Malicious trigger- removal Confluences on i3 public nodes Attack Defense

30 Outline Implementation Examples Security  Applications  Protection against DoS attacks –Service composition platform Legacy application support

31 In a Nutshell Problem scenario: attacker floods the incoming link of the victim Solution: stop attacking traffic before it arrives at the incoming link –Today: call the ISP to stop the traffic, and hope for the best! Our approach: give end-host control on what packets to receive –Enable end-hosts to stop the attacks in the network

32 Why End-Hosts (and not Network)? End-hosts can better react to an attack –Aware of semantics of traffic they receive –Know what traffic they want to protect End-hosts may be in a better position to detect an attack –Flash-crowd vs. DoS

33 Some Useful Defenses 1.White-listing: avoid receiving packets on arbitrary ports 2.Traffic isolation: –Contain the traffic of an application under attack –Protect the traffic of established connections 3.Throttling new connections: control the rate at which new connections are opened (per sender)

34 1. White-listing Packets not addressed to open ports are dropped in the network –Create a public trigger for each port in the white list –Allocate a private trigger for each new connection

35 2. Traffic Isolation Drop triggers being flooded without affecting other triggers –Protect ongoing connections from new connection requests –Protect a service from an attack on another services Victim (V) Attacker (A) Legitimate client (C) ID 2 V ID 1 V Transaction server Web server

36 2. Traffic Isolation Drop triggers being flooded without affecting other triggers –Protect ongoing connections from new connection requests –Protect a service from an attack on another services Victim (V) Attacker (A) Legitimate client (C) ID 1 V Transaction server Web server Traffic of transaction server protected from attack on web server Traffic of transaction server protected from attack on web server

37 Server (S) Client (C) 3. Throttling New Connections Redirect new connection requests to a gatekeeper –Gatekeeper has more resources than victim –Can be provided as a 3 rd party service ID C C XS puzzle puzzle’s solution Gatekeeper (A) ID P A

38 Outline Implementation Examples Security Architecture Optimizations  Applications –Protection against DoS attacks  Service composition platform Legacy application support

39 Service Composition Goal: allow third-parties and end-hosts to easily insert new functionality on data path –E.g., firewalls, NATs, caching, transcoding, spam filtering, intrusion detection, etc.. Why i3? –Make middle-boxes part of the architecture –Allow end-hosts/third-parties to explicitly route through middle-boxes

40 Example Use Bro system to provide intrusion detection for end-hosts that desire so M client A server B i3 Bro (middle-box) id M M id BA B id AB A (id M :id BA, data) (id BA, data) (id M :id AB, data) (id AB, data)

41 Outline Implementation Examples Security Applications  Legacy Application Support

42 The Problem New network architectures & overlays provide new features –RON : resilience to path failures –i3 : mobility, NAT traversal, anycast, multicast –…–… But still no widespread deployment How take advantage of new functionality? Enable popular applications (ssh, Firefox, IE) to benefit from new features

43 Solutions Approach 1: rewrite/port apps for each new overlay –time-consuming, tedious, impossible for closed source apps Approach 2: enable support for legacy applications over multiple overlays We chose approach 2!

44 Overlay Convergence Architecture for Legacy Applications (OCALA) Legacy Applications (ssh, firefox, explorer, …) Legacy Applications (ssh, firefox, explorer, …) Transport Layer (TCP, UDP, …) Transport Layer (TCP, UDP, …) IP Layer Interpose an Overlay Convergence Layer between transport layer and overlay networks

45 Overlay Convergence Architecture for Legacy Applications (OCALA) Overlay Convergence (OC) Layer Overlay (DOA, DTN, HIP, i3, RON, …) Overlay (DOA, DTN, HIP, i3, RON, …) Legacy Applications (ssh, firefox, explorer, …) Legacy Applications (ssh, firefox, explorer, …) Transport Layer (TCP, UDP, …) Transport Layer (TCP, UDP, …) OC Independent (OC-I) Sublayer OC Dependent (OC-D) Sublayer Interpose an Overlay Convergence Layer between transport layer and overlay networks

46 Simultaneous access to multiple overlays OC-I i3 Firefox OC-I RON ssh RON IRC ssh … OC-D i3 RON Internet … OC-I i3 IRC … Host A Host B Host C IP

47 Which overlay to use? IP address and port number : –E.g.: Forward all packets sent to port 22 over i3 DNS name: –E.g.: Forward all packets sent to odu.edu.ron over RON (Resilient Overlay Networks) –E.g.: Forward all packets sent to odu.edu.i3 over i3

48 Bridging Multiple Overlays Communication across overlays Stitch together functionality OC-I Host A Appl. OC-I Host C (foo.ron) Appl. OC-I Host B (bar.i3) i3 OC-D i3RON i3 RON tunnel path

49 Legacy Client Gateways – Demo Clients need not run OCALA locally Gateway has special Legacy Client IP (LCIP) module OC-I Appl. OC-I LCIP OV Legacy gateway i3 Internet Legacy Client OV Overlay server (dilip.i3) DNSreq(ionhome.pli3.ocalaproxy.net)

50 Legacy Client Gateways – Demo Access the following URL using your web browser:

51 Status i3 available as a service on Planetlab OCALA available on Linux, Window/XP; enable legacy applications to take advantage of –i3 –RON –HIP (Host Identity Protocol) –… Current applications –Mobility –Transparent access to machines behind NATs –Secure and transparent access to services behind firewalls

52 Conclusions Indirection – key technique to implement basic communication abstractions –Multicast, Anycast, Mobility, … Internet Indirection Infrastructure –Platform to deploy new services in the Internet –Highly flexible forwarding infrastructure: allows end- hosts and third-parties to choose their own routes! i3: OCALA: i3: OCALA:

53 Other Architecture Optimizations Shortcuts: –Sender can cache receiver instead of an i3 server –Send subsequent packets directly to the receiver Private trigger aggregation –Have all private triggers share the same prefix –Insert only an anycast trigger with that prefix in i3

54 Conclusions i3: tremendously flexible infrastructure –Allow end-hosts to control forwarding and replication points in the infrastructure –Abstract away the behavior of hosts (e.g., mask mobility) form each other –Provide a global and flexible identifier space IDs can identify virtually anything, end-hosts, services, sessions, humans, devices, etc..

55 Enable Many Applications Access to machines behind NATs Transparent access to services/machines behind firewalls –Like VPNs but more secure and flexible Protection against DoS attacks Anycast Transparent switching between two interfaces …

56 Status Code publicly available: –Supports Linux & Windows XP/2000 legacy applications –Includes a light weight Chord implementation

57 Private Trigger Aggregation (Problem) Each host maintains a private trigger in i3 for each other host it communicate with Too many private triggers –E.g., can reach 100 after browsing for 5 min! Every packet goes through an i3 server even if users don’t want this A B C D id AB A id AC A id AD A (id AB,data) (id AC,data) (id AD,data)

58 Private Trigger Aggregation (Solution) Chose all private triggers to have the same prefix A Keep only an anycast trigger with prefix A in infrastructure A B C D id A * A (id AC,data) (id AD,data) (id AB,data)

59 Private i3 Nodes (Problem) The resilience and truthworthiness of i3 infrastructure may not be good enough for various users –E.g., a company using i3 to provide transparent and secure access for remote workers

60 Private i3 Nodes (Solution) Allow customers to add their i3 nodes to infrastructure using anycast and shortcuts ACME Inc id A* A S id AS A (id AS, data) cache ACME i3 server

61 Service Composition: Research Questions How to locate and compose services? How to distribute state across hosts and middle-boxes? How to transparently recover when a middle- box fails? How is the state recovered?

62 Shortcuts (Problem) Each packet is traversing an i3 server no matter whether the user wants/needs to or not

63 Shortcuts (Solution) Associate an “allow-caching” bit with both packets and triggers If both the packet and trigger have the bit set  sender can cache receiver’s address LALA LBLB id BA A 1 Host A (IP B )Host B 3 1 (id BA,data) cache=IP B 2

64 Shortcuts: Discussion Shortcuts are allowed only if both the sender and receiver agree i3 used as a lookup infrastructure Indirection still needed for –Mobility –Symmetric NAT & Firewall traversal –Anonymity