Bob Knowledge Plane -- Scaling of the WHY App Bob Braden, ISI 24 Sept 03

Bob Scaling [How] can we make KP services "scalable" (whatever that means)? –Network traffic –Processing –Storage E.g., suppose that every end system uses WHY. –This should be a good example of scaling issues in the KP To diagnose the cause of failure, typically need information that is available only in neighborhood of failure => wide-area problem.

Bob IP Path Diagnosis Consider a subset of the WHY problem: diagnosis of an IP data path: –Can S send IP datagram to D, and if not, why not? For the "cause", simply tell which node or link is broken. –Thus, ask for information currently provided by traceroute.

Bob A Simple, Analogous Scaling Problem Let’s think about a connectivity testing tool that runs in the data/control-plane (not the KP): IPdiagnose. –Operates like traceroute, hop/hop along data path. –Returns path vector: list of router hops to failure point. Want to make IPdiagnose scalable, in case all the users trigger it. Purpose: –Insight into more general KP scaling –Insight into DDC's model

Bob Possible Approaches to IPdiagnose 1.Using vanilla traceroute OH ~  Ne   Path length (number of hops)  Diagnostic frequency (WHY requests per sec per end node) Ne = number of end nodes that issue traceroutes. 2.Record-&-Return-Route (RRR) msg, processed in each router. OH ~  Ne D message S3 2 1 X S,1>D S,1,2,3>D S>D S,1,2>D

Bob Make it Scalable Lower the overhead by decreasing average path length  –Move (prior) results as close to end points as possible/practicable. This reduces the diagnostic traffic in the center of the network. –To achieve this, use: 3. "Aggregation" –If matching request for same D arrives while previous is pending, hold it and satisfy when reply comes back. 4. Demand-driven result caching –Cache results back along the path from S; –Use cached results to satisfy subsequent requests for same destination that come later.

Bob Result Caching Search messages from S gather path vector in forward direction. Return messages visit each node along return path to S and leave IPdiagnose result state there. {2,3>D} {3>D} {>D} State retained in node 1: If a later IPdiagnose S'->D reaches this node, return the path {S’,…,1, 2, 3 > D} SD3 2 1 X S,1>D S,1,2>D S,12,3>D message S'

Bob This is not quite certain… Note: cached path could be unreliable, but it generally works in the absence of policy routing. SD3 2 1 X S,1>D S,1,2>D 2’ D’

Bob More Scalability Suppose have cached state for failed path S -> D. Does this help for another path S' -> D' that shares 1, 2, 3, …? –Suppose that routing in node 3 supplies an address range Dr that contains address D. –Cached results can contain Dr. –If D’ is contained in Dr, then node 1 can use cached state {2,3>Dr} to infer broken path {1,2,3>D’}. SD3 2 1 X {2,3>Dr} {3>Dr} D’ S’ Dr

Bob Flushing the Cache New requests matching cache inhibit timeout. Some percent of matching requests will be forwarded anyway, as probe requests. A node will initiate a reverse message towards all relevant senders to adjust/remove cached state, if: –Routing changes Dr, or –A probe request finds next hop info that differs from cached path.

Bob Relation to DDC Model in KP Dest address D is the (only) variable of the “tuple” composing the request. Forwarding is not offer-based (unless next-hop routing calculation is considered an “offer”) Does not exactly match DDC's “Aggregation” story (?) –First request arrives: Don’t want to delay it to await a matching request, so cache and forward it. Is this an "aggregation"? DDC's model does not have result caching. In KP, must consider complexity caused by regions. –Sparse overlay mesh of TPs

Bob Other Approaches to IPdiagnose 4. Flooding (unconstrained diffusion) –Every diagnostic event (link-down event) is flooded out to edges, where it matches requests. –I am confused about scalability here. Intuitively this seems unscalable, but I don’t see how to justify that. Flooding cost ~ O(#links * #faults) (one per fault per link) Request cost ~ O(  * Ne ) (path length = 1)

Bob More Approaches to IPdiagnose 5. Directed Diffusion –Link state changes are flooded out towards edges in directions of significant fluxes of incoming WHY requests. –In sparse directions, use RRR messages or result- caching within the network, as discussed earlier. –This is reverse of Clark’s proposal – here the requests are creating a gradient to control the diffusion of satisfactions nearer to the users.

Bob (The End)

Bob Demand-Driven Result Caching Creates a depth-first diffusion of IPdiagnose replies, triggered by requests for the same destination that share part of the same path. Note that if path is not in fact broken, then nothing is cached and then scaling of IPdiagnose stinks.

Bob DDC’s Request Satisfaction Model Route a request hop/hop (roughly) paralleling the data path to reach a Request Satisfier (RS) near failure node F. Satisfaction: IP path vector from S to F. Recursive induction step at node K (Assume RS is in each node) : –Request "(IPFAIL, D, (S, N 1,…N n ))" arrives at node N n. –Analysis: “S cannot send datagrams to D, but packets from S to D reach me.. The next-hop node towards D from is N n+1. I will test whether I can get to N n+1 and, if so, pass request "(IPFAIL, D, (S, N 1, … N n+1 ) along to it. If not, I will return path vector (S, N 1, … N n ) back to S."

Bob DDC’s Model … More complex version of model: take into account the region structure of Internet. E.g., RS per region. Request arrives at RSn; induction step of analysis is: “Packets from S to D reach my (entry) edge node E n.” “I have evidence that packets are flowing from En to my appropriate (exit) edge node E’ n. The next-hop RS, in the next AS along the data path towards D, is RS n+1 and the next hop towards D from E’ n is E n+1. I will test whether I can get to E n+1, and if so, pass this request along to RS n+1, else I will return path vector (S, N 1, … E n, … E’ n ) to S.”

Bob Result Caching … general case X ||S|| Any source S  ||S|| uses the broken link to reach any D  ||D|| ||D|| Infer ||D|| from routing, Store information about broken link to ||D|| near every S  ||S|| INTERNET