LISP Locator Reachability Algorithms Dino Farinacci, Dave Meyer, Darrel Lewis, Vince Fuller, Andrew Partan, Noel Chiappa IETF Stockholm LISP Working Group July 2009

RLOC Reach Algorithms July 2009 Slide 2 Agenda Problem Statement Observe Data Path Combinations Using TCP Heuristics –“TCP-counts” Using data-plane echoing –“echo-nonces” All in Unison? Implementation Report

RLOC Reach Algorithms July 2009 Slide 3 Problem Statement ITR A needs to know if RLOC B is reachable ITR A needs to know when it can switchover to B’ ITR A cannot depend on a B-prefix route to determine if RLOC B is reachable A’A’ A’A’ A A B’B’ B’B’ B B SD ?

RLOC Reach Algorithms July 2009 Slide 4 Problem Statement When ITR B sends to RLOC A, doesn’t mean that ITR A can reach RLOC B All you know is that RLOC B has not crashed but don’t know the forward-path from A -> B A’A’ A’A’ A A B’B’ B’B’ B B SD ?

RLOC Reach Algorithms July 2009 Slide 5 Problem Statement Loc-reach-bits from ITR B to RLOC A just tells A that RLOC B’ is not down Does not tell you that path from ITR A to RLOC B’ is reachable A’A’ A’A’ A A B’B’ B’B’ B B SD 0x

RLOC Reach Algorithms July 2009 Slide 6 Solution Space Have each ITR probe each ETR for every map-cache entry –Can be done with control messaging –Can be piggybacked with data Use deep-packet-inspection heuristics Can’t use a database for up/down status –Reachability is relative to the source –Up/down status only useful when ETR is down –Up status tells you can test the path Send and pray –Use ICMP Unreachables to tell you path down status –But there is no ICMP mechanism to tell you when back up

RLOC Reach Algorithms July 2009 Slide 7 Dilemma Need to detect quickly when RLOC is down –So you can switchover fast Need to have recent up status for an RLOC –So you can switch to a working path Existence of a route to RLOC doesn’t give you up status –Must use a keepalive mechanism –Should have up status for using RLOC “N times M” control messaging doesn’t scale –Especially if you want to switchover fast –Tradeoff message overhead versus fast convergence

RLOC Reach Algorithms July 2009 Slide 8 Possible Data Paths Totally symmetric –ITR A and B see each other as up due to receipt of data –They could use piggyback keepalives to determine forward- path is up –But if there is no site-sourced data offered to ITRs they have to take a leap of faith the forward-path is up A’A’ A’A’ A A B’B’ B’B’ B B SD

RLOC Reach Algorithms July 2009 Slide 9 Possible Data Paths Source symmetric A’A’ A’A’ A A B’B’ B’B’ B B SD A’A’ A’A’ A A B’B’ B’B’ B B SD Return path symmetric Same case - ITR sends to one RLOC but may only receive from other RLOC

RLOC Reach Algorithms July 2009 Slide 10 Possible Data Paths Totally asymmetric - “the square” –Each xTR only has send *or* receive information –xTRs at a site don’t synchronize state –Piggyback keepalives can’t work –ITR A could request an echo in data-plane but B must reply with a control message –ITR A would have to keepalive with control message to B’ and B’ would reply with a control message A’A’ A’A’ A A B’B’ B’B’ B B SD

RLOC Reach Algorithms July 2009 Slide 11 DPI Mechanism - “TCP-counts” Use a TCP connection setup heuristic –Specifically design for “the square” –ITRs count SYNs-sent and ACKs-sent for all connections –If ACKs are being sent, the return path from B’ to A’ is working *and* therefore path from A to B is working –If SYNs are sent but no ACKs then there is no return traffic –But A -> B could be working when B -> D, D -> B’, B’ -> A’, or A’ -> S is broken, in this case A should not switchover to B’ –This mechanism gives you path up status but not good down status A’A’ A’A’ A A B’B’ B’B’ B B SD SYN SYN/ACK ACK SYN SYN/ACK ACK

RLOC Reach Algorithms July 2009 Slide 12 Piggybacking - Echo Nonce Nonce in data packet –ITR requests ETR to echo nonce back –Request sets E-bit in data packet –Echo from ETR contains ITR’s nonce with E-bit clear –Tests if forward-path is up –Only works when symmetric (bidirectional traffic) between RLOC pairs –Detect down status via timeout of echo-nonce –Can be quicker convergence than control message keepalive as long as data is sent from ITR to ETR A’A’ A’A’ A A B’B’ B’B’ B B SD E=1, nonce: 0x E=0, nonce: 0x

RLOC Reach Algorithms July 2009 Slide 13 LISP Header Format Documented in draft-ietf-lisp-03.txt Piggybacking - Echo Nonce

RLOC Reach Algorithms July 2009 Slide 14 Implementation - “TCP-counts” Cost in memory –2 integers per RLOC per map-cache entry Data-plane counts syns-sent and acks-sent during ITR encapsulation –Unilateral algorithm Every minute control-plane looks at counts –(!loc->syns_sent && !loc->acks_sent) -> RLOC is idle, leave in up state –(!loc->syns_sent && loc->acks_sent) -> RLOC is up, leave in up state –(loc->syns_sent && loc->acks_sent) -> RLOC is up, leave in up state –(loc->syns_sent && !loc->acks_sent) -> RLOC went down, take down If down, bring up if packet received –Not square data path anymore If down, after 3 minutes, bring back up to start counting

RLOC Reach Algorithms July 2009 Slide 15 Implementation - Echo Nonce Cost in memory –6 integers per RLOC per map-cache entry –ITR to ETR direction (ETR is echoing) (1) Next echo-nonce request to send (2) Last remote echo-nonce received (3) Packets receive count while in echo-nonce request state (4) Timestamp when first entering echo-nonce request state –ETR to ITR direction (ITR is echoing) (5) Last remote echo-nonce request received (6) Next echo-nonce to send

RLOC Reach Algorithms July 2009 Slide 16 Implementation - Echo Nonce Handle collision –Both sides could be in echo-nonce request state at the same time –They would never echo each other 2 mechanisms to avoid collision: –Use RLOC addressing as tiebreaker Force higher RLOC address to be in echo-nonce request state After lower RLOC address echos can enter echo-nonce request state –Use nanosecond clock Odd you keep requesting Even you start echoing

RLOC Reach Algorithms July 2009 Slide 17 Every minute enter echo-nonce request state if data received in last minute –Exit when nonce is echoed or one minute has elapsed, former stays up, later take down –If no packets received keep up –Wait for arrival of 10 packets (within the minute interval) before checking if nonce was echoed If nonce sent in request does not match any echoed nonces, take RLOC down When down for 3 minutes, bring up and enter echo- nonce request state If down and receive packet, bring up and enter echo- nonce request state Implementation - Echo Nonce

RLOC Reach Algorithms July 2009 Slide 18 When receiving an echo-nonce request –Within 15 seconds return echo for nonce –Continue returning echo up to 1 minute –Accept request regardless if RLOC is up or down Some good news –Implementation doesn’t do source RLOC lookup in data-plane –Part of statistics processing allows echo-nonce state to be conveyed from data-plane to control-plane Some bad news –Easy to explain at protocol level –Hard to implement due to a lot of control-plane/data-plane interaction Implementation - Echo Nonce

RLOC Reach Algorithms July 2009 Slide 19 All in Unison? Echo-nonce doesn’t make you depend on TCP TCP-counts can help echo-nonce with the asymmetric data paths When one says down and the other says up, keep RLOC up When both say down, take RLOC down even when loc- reach-bits say up - because path is down When loc-reach-bits say down, take RLOC down Conclusion –Loc-reach-bits tell you when hard failure is close to RLOC –Echo-nonce and tcp-counts tell you about path failure

RLOC Reach Algorithms July 2009 Slide 20 Unidirectional Data? When unidirectional data occurs among sites (to one or more ETRs) –ITRs can’t tell if forward-path is up –Echo noncing won’t work –TCP-counts won’t work ITR must send a Map-Request to the RLOC –Used as a control message keepalive –Don’t spec this yet - maintain resistance This will be the default scenario for PTRs –Since it only encapsulates data packets When all priority 1 RLOCs are down and using priority 2 RLOCs –Can send Map-Request to priority 1 RLOCs to test path before using them –Make-before-Break at the expense of control message overhead –Don’t spec this yet - maintain resistance

RLOC Reach Algorithms July 2009 Slide 21 Summary The urge to get this 100% right will cause scalability problems –Resist the urge Will the imperfection will be okay? –We are solving most of the problem With active-active multi-homing life is good –We’ll have symmetric paths A’A’ A’A’ A A B’B’ B’B’ B B SD

RLOC Reach Algorithms July 2009 Slide 22 How to Configure Defaults to neither enabled lisp loc-reach-algorithm {echo-nonce | count-tcp} Debug command debug lisp loc-reach-algorithm Show command show {ip | ipv6} lisp map-cache Supported in release dino-lisp-126

RLOC Reach Algorithms July 2009 Slide 23 Sample Output TCP-counts SYN and ACK counters /16, uptime: 00:06:09, expires: , via static State: complete, last modified: 00:06:09, map-source: local Locator Uptime State Priority/Weight Packets In/Out :06:09 up 1/50 16/17 Last up/down state change: 00:06:09 Last data packet in/out: 00:01:08/00:01:08 Last control packet in/out: never/never Last priority/weight change: never/never TCP-counts loc-reach algorithm: SYNs sent: 9, ACKs sent: :06:09 up 1/50 0/0 Last up/down state change: 00:06:09 Last data packet in/out: never/never Last control packet in/out: never/never Last priority/weight change: never/never TCP-counts loc-reach algorithm: SYNs sent: 2, ACKs sent: 0 RLOC staying up RLOC going down

RLOC Reach Algorithms July 2009 Slide 24 Sample Output When in echo-nonce request state /16, uptime: 00:44:49, expires: , via static State: complete, last modified: 00:44:49, map-source: local Locator Uptime State Priority/Weight Packets In/Out :44:49 up 1/50 0/0 Last state change: 00:44:49 Last data packet in/out: never/never Last control packet in/out: never/never Last priority/weight change: never/never Echo-nonce loc-reach algorithm: Next request nonce to RLOC: 0x0049d55c Last echoed nonce from RLOC: 0x Packets from RLOC in echo-nonce request state: 0 Last request nonce from RLOC: 0x Next echo nonce to RLOC: 0x :44:49 up 2/50 0/0 Last state change: 00:44:49 Last data packet in/out: never/never Last control packet in/out: never/never Last priority/weight change: never/never Echo-nonce loc-reach algorithm: Next request nonce to RLOC: 0x0049fb90 Last echoed nonce from RLOC: 0x Packets from RLOC in echo-nonce request state: 0 Last request nonce from RLOC: 0x Next echo nonce to RLOC: 0x

RLOC Reach Algorithms July 2009 Slide 25 Sample Output When echoing a nonce /16, uptime: 00:56:05, expires: , via static State: complete, last modified: 00:56:05, map-source: local Locator Uptime State Priority/Weight Packets In/Out :56:05 up 1/50 1/2 Last state change: 00:56:05 Last data packet in/out: 00:00:03/00:00:03 Last control packet in/out: never/never Last priority/weight change: never/never Echo-nonce loc-reach algorithm: Next request nonce to RLOC: 0x0049d55c Last echoed nonce from RLOC: 0x Packets from RLOC in echo-nonce request state: 1 Last request nonce from RLOC: 0x00f7aa95 Next echo nonce to RLOC: 0x00f7aa :56:05 up 2/50 0/0 Last state change: 00:56:05 Last data packet in/out: never/never Last control packet in/out: never/never Last priority/weight change: never/never Echo-nonce loc-reach algorithm: Next request nonce to RLOC: 0x0049fb90 Last echoed nonce from RLOC: 0x Packets from RLOC in echo-nonce request state: 0 Last request nonce from RLOC: 0x Next echo nonce to RLOC: 0x

RLOC Reach Algorithms July 2009 Slide 26 Big Question Can control message probing scale? Stay tuned

RLOC Reach Algorithms July 2009 Slide 27