Y. WuHotNets-XII (Nov 22, 2013)1 Answering Why-Not Queries in Software-Defined Networks with Negative Provenance Yang Wu* Andreas Haeberlen* Wenchao Zhou + Boon Thau Loo* * University of Pennsylvania + Georgetown University

-SDN gives power, but they can have bugs -need good debuggers for SDNs Y. WuHotNets-XII (Nov 22, 2013)2 Motivation: SDN debugging Internet HTTP Server Data Center Network Controller

Why is the HTTP server getting DNS queries? -existing tools produce backtraces Y. WuHotNets-XII (Nov 22, 2013)3 Motivation: SDN debugging Internet HTTP Server Data Center Network Controller DNS Packet DNS Packet arrived at HTTP Server DNS Packet received at Switch Broken FlowEntry existed at Switch … … … Program DNS Packet Broken FlowEntry

Why is the HTTP server getting DNS queries? Y. WuHotNets-XII (Nov 22, 2013)4 Motivation: SDN debugging Internet HTTP Server Data Center Network Controller DNS Packet arrived at HTTP Server DNS Packet received at Switch FlowEntry existed at Switch … … DNS Packet Broken FlowEntry -backtraces require an observed event as starting point

-no starting point when debugging missing events -problem: diagnose missing events in networks Y. WuHotNets-XII (Nov 22, 2013)5 Challenge: no starting point Internet HTTP Server Data Center Network Controller Why is the HTTP server NOT getting requests?

Y. WuHotNets-XII (Nov 22, 2013)6 Approach: counter-factual reasoning -basis: counter-factual reasoning Find all the ways a missing event could have occurred, and show why each of them did not come to pass.

Y. WuHotNets-XII (Nov 22, 2013)7 Approach: counter-factual reasoning -basis: counter-factual reasoning NYC DC Why did Bob NOT arrive at DC?

Y. WuHotNets-XII (Nov 22, 2013)8 Approach: counter-factual reasoning -a SDN debugger that answers “Why-Not” questions Internet HTTP Server Data Center Network Controller No HTTP Packet arrived at HTTP Server No Forwarding-FlowEntry installed at Switch HTTP Packet received at Switch Dropping-FlowEntry existed at Switch … … Program … ??? HTTP Packet Dropping- FlowEntry Why is the HTTP server NOT getting requests?

Y. WuHotNets-XII (Nov 22, 2013)9 Approach: counter-factual reasoning No HTTP Packet arrived at HTTP Server No Forwarding-FlowEntry installed at Switch HTTP Packet received at Switch Dropping-FlowEntry existed at Switch … … Negative Provenance Graph

10 Overview Solution Status Implementation Fast queries using R-trees Supporting legacy programs Challenge: No starting point Goal: Diagnose missing events WHY NOT ? Approach: Counter-factual reasoning Generating negative provenance Improving readability Background: Provenance

Y. WuHotNets-XII (Nov 22, 2013)11 Background: network datalog -network datalog (NDLOG) PacketSent :- FlowEntry, PacketReceived. Head Predicate

Why is the HTTP server getting DNS queries? -provenance Y. WuHotNets-XII (Nov 22, 2013)12 Background: provenance DNS Packet arrived at HTTP Server DNS Packet received at Switch FlowEntry existed at Switch … … APPEAR(DNS Server, t=3pm) t=[1pm,now]) APPEAR(DNS t=3pm) EXIST INSERT, DELETE DERIVE, UNDERIVE APPEAR, DISAPPEAR SEND, RECEIVE DELAY

Y. WuHotNets-XII (Nov 22, 2013)13 Negative provenance model -we extend the model to support negative provenance EXIST INSERT, DELETE DERIVE, UNDERIVE APPEAR, DISAPPEAR SEND, RECEIVE DELAY NEXIST NINSERT, NDELETE NDEDRIVE, NUNDERIVE NAPPEAR, NDISAPPEAR NSEND, NRECEIVE NARRIVE +

Y. WuHotNets-XII (Nov 22, 2013)14 Negative provenance model No HTTP Packet arrived at HTTP Server No Forwarding-FlowEntry installed at Switch HTTP Packet received at Switch Dropping-FlowEntry existed at Switch … … -we extend the model to support negative provenance Why is the HTTP server NOT getting requests? NAPPEAR(HTTP Server, t=[-∞, 3pm]) t=[-∞, 3pm]) t=[10am,now]) APPEAR(HTTP t=2pm)

Y. WuHotNets-XII (Nov 22, 2013)15 Generating negative provenance graphs -pseudo-code for building negative provenance graph

Y. WuHotNets-XII (Nov 22, 2013)16 Generating negative provenance graphs -explain non-existence of head with missing predicates FlowEntry PacketSent :- FlowEntry, PacketReceived. PacketReceived NEXT(PacketSent, t=[t1,now]) ??? time t1 t2 t3 t4 t5 now Head Predicate

Y. WuHotNets-XII (Nov 22, 2013)17 Generating negative provenance graphs -explain non-existence with missing pre-conditions PacketSent FlowEntry PacketReceived NEXT(PacketSent, t=[t1,now]) time t1 t2 t3 t4 t5 now NEXT(PacketReceived, t=[t1,t2]) NEXT(FlowEntry, t=[t2,t3]) NEXT(PacketReceived, t=[t3,t4]) NEXT(FlowEntry, t=[t2,t3]) NEXT(PacketReceived, t=[t5,now])

Y. WuHotNets-XII (Nov 22, 2013)18 Generating negative provenance graphs -select simple explanations time FlowEntry PacketReceived NEXT(PacketSent, t=[t1,now]) NEXT(FlowEntry, t=[t1,now])

Y. WuHotNets-XII (Nov 22, 2013)19 Generating negative provenance graphs -select simple explanations -finding the simplest explanation is as least as hard as Set Cover (NP-Hard) -close approximation is good enough time FlowEntry PacketReceived PacketSent :- FlowEntry, PacketReceived. Head Predicate

Y. WuHotNets-XII (Nov 22, 2013)20 Generating negative provenance graphs -pseudo-code for building negative provenance graph

Y. WuHotNets-XII (Nov 22, 2013)22 Improving readability -prune branches that correspond to failed assertions … NEXIST(FlowEntry(Pri > 255))... assert(FlowEntry(Pri < 63)) … … … Negative Provenance Graph Controller Program

Y. WuHotNets-XII (Nov 22, 2013)23 Improving readability Prune logical inconsistencies. Prune failed assertions. Branch coalescing. Application-specific invariants. Hide transient event chains. Summarize super-vertex.

Y. WuHotNets-XII (Nov 22, 2013)25 Improving readability -typical absolute size < 20 -~90% reduction in graph size

26 Overview Solution Challenge: No starting point Goal: Diagnose missing events WHY NOT ? Approach: Counter-factual reasoning Generating negative provenance Improving readability Background: Provenance Status Implementation Fast queries using R-trees Supporting legacy programs

Y. WuHotNets-XII (Nov 22, 2013)27 Status: implementation -graph builder -event storage -Mininet Internet HTTP Server Data Center Network (Mininet) SDN Controller Event Storage Graph Builder

Y. WuHotNets-XII (Nov 22, 2013)28 Status: fast queries -optimization: R-Tree Internet HTTP Server Data Center Network (Mininet) SDN Controller Event Storage Graph Builder

Y. WuHotNets-XII (Nov 22, 2013)29 Status: supporting legacy -Frenetic frontend Internet HTTP Server Data Center Network (Mininet) SDN Controller Event Storage Graph Builder

Y. WuHotNets-XII (Nov 22, 2013)30 -Goal: a debugger for missing events in SDNs Example: Why HTTP server is not getting requests? -Approach: Negative Provenance Use counterfactual reasoning to explain how the missing events could have occurred. -Raw provenance graphs are complex, but can be simplified considerably using heuristics Encouraging results from case studies. -We are working on a prototype implementation Efficient queries due to a special index. Supports an existing SDN language (Frenetic). Questions?