1. Declarative Networking: Extensible Networks with Declarative Queries Boon Thau Loo University of California, Berkeley

2. Era of change for the Internet in the thirty-odd years since its invention, new uses and abuses, …., are pushing the Internet into realms that its original design neither anticipated nor easily accommodates….. Overcoming Barriers to Disruptive Innovation in Networking, NSF Workshop Report 05

3. Efforts at Internet Innovation Evolution: Overlay Networks Commercial (Akamai, VPN, MS Exchange servers) P2P (filesharing, telephony) Research prototypes on testbed (PlanetLab) Revolution: Clean slate design NSF Future Internet Design (FIND) program NSF Global Environment for Network Investigations (GENI) initiative Missing: software tools that can significantly accelerate Internet innovation Internet Overlay

4. Approach: Declarative Networking A declarative framework for networks: Declarative language: ask for what you want, not how to implement it Declarative specifications of networks, compiled to distributed dataflows Runtime engine to execute distributed dataflows Observation: Recursive queries are a natural fit for routing

5. P2 Declarative Networking System Network Specifications as Queries Query Planner Dataflow Engine Network Protocols http://p2.cs.berkeley.edu Dataflow

6. The Case for Declarative Ease of programming: Compact and high-level representation of protocols Orders of magnitude reduction in code size Easy customization Safety: Queries are sandboxed within query processor Potential for static analysis techniques on safety What about efficiency? No fundamental overhead when executing standard routing protocols Application of well-studied query optimizations Note: Same question was asked of relational databases in the 70s.

7. Main Contributions Declarative Routing [HotNets 04, SIGCOMM 05]: Extensible Routers (balance of flexibility, efficiency and safety). Declarative Overlays [SOSP 05]: Rapid prototyping of new overlay networks Database Fundamentals [SIGMOD 06] : Network specific query language and semantics Distributed recursive query execution strategies Query Optimizations, classical and new

8. A Breadth of Use Cases Implemented to date: Textbook routing protocols (3-8 lines, UCB/Wisconsin) Chord DHT overlay routing (47 lines, UCB/IRB) Narada mesh (16 lines, UCB/Intel) Distributed Gnutella/Web crawlers (Dataflow, UCB) Lamport/Chandy snapshots (20 lines, Intel/Rice/MPI) Paxos distributed consensus (44 lines, Harvard) In Progress: OSPF routing (UCB) Distributed Junction Tree statistical inference (UCB)

9. Outline Background The Connection: Routing as a Query Execution Model Path-Vector Protocol Example Query specification protocol implementation More Examples Realizing the Connection P2: Declarative Routing Engine Beyond routing: Declarative Overlays Conclusion

10. Traditional Router Packets Traditional Router Control Plane Forwarding Plane Routing Protocol Neighbor Table updates Forwarding Table updates

11. Review: Path Vector Protocol Advertisement: entire path to a destination Each node receives advertisement, add itself to path and forward to neighbors path=[c,d]path=[b,c,d]path=[a,b,c,d] c advertises [c,d]b advertises [b,c,d] bdca

12. Declarative RouterTraditional Router Declarative Router Declarative Queries Control Plane Forwarding Plane Packets P2 Engine Routing Protocol Neighbor Table updates Forwarding Table updates

13. Introduction to Datalog,, …,. Datalog rule syntax: Types of conditions is body: Input tables: link(src,dst) predicate Arithmetic and list operations Head is an output table Recursive rules: result of head in rule body Body Head

14. All-Pairs Reachability R2: reachable(S,D) link(S,Z), reachable(Z,D) R1: reachable(S,D) link(S,D) Input: link(source, destination) Output: reachable(source, destination) For all nodes S,D, If there is a link from S to D, then S can reach D. link(a,b) – there is a link from node a to node b reachable(a,b) – node a can reach node b

15. All-Pairs Reachability R2: reachable(S,D) link(S,Z), reachable(Z,D) R1: reachable(S,D) link(S,D) Input: link(source, destination) Output: reachable(source, destination) For all nodes S,D and Z, If there is a link from S to Z, AND Z can reach D, then S can reach D.

16. Towards Network Datalog Specify tuple placement Value-based partitioning of tables Tuples to be combined are co-located Rule rewrite ensures body is always single-site All communication is among neighbors No multihop routing during basic rule execution Enforced via simple syntactic restrictions

17. All-Pairs Reachability R1: reachable(@S,D) link(@S,D) R2: reachable(@S,D) link(@S,Z), reachable(@Z,D) Network Datalog Query: reachable(@M,N) @SD @ab c d reachable Output table: Input table: Query: reachable(@a,N) @SD @cb d link @SD @bc a link @SD @ab link @SD @dc link bdc a @SD @ba c d reachable @SD @ca b d reachable @SD @da b c reachable Location Specifier @S Query: reachable(@a,N)

18. Path Vector in Network Datalog Input: link(@source, destination) Query output: path(@source, destination, pathVector) R1: path(@S,D,P) link(@S,D), P=(S,D). R2: link(@Z,S), path(@S,D,P) P=S P 2. path(@Z,D,P 2 ), Query: path(@S,D,P) Add S to front of P 2

19. R1: path(@S,D,P) link(@S,D), P=(S,D). R2: path(@S,D,P) link(@Z,S), path(@Z,D,P 2 ), P=S P 2. @SDP DP @cd[c,d] Query Execution @SDP DP Query: path(@a,d,P,C) Neighbor table: @SD @cb d link @SD @bc a link @SD @ab link @SD @dc link bdca path Forwarding table:

20. @SDP DP DP @cd[c,d] Query Execution Forwarding table: @SDP @bd[b,c,d] bdca path(@b,d,[b,c,d]) R1: path(@S,D,P) link(@S,D), P=(S,D). R2: path(@S,D,P) link(@Z,S), path(@Z,D,P 2 ), P=S P 2. Query: path(@a,d,P,C) Neighbor table: @SD @cb d link @SD @bc a link @SD @ab link @SD @dc link path @SDP @ad[a,b,c,d] path(@a,d,[a,b,c,d]) Communication patterns are identical to those in the actual path vector protocol Matching variable Z = Join

21. Sanity Check All-pairs shortest latency path query: Query convergence time: proportional to diameter of the network. Same as hand-coded PV. Per-node communication overhead: Increases linearly with the number of nodes Same scalability trends compared with PV/DV protocols

22. Outline Background The Connection: Routing as a Query Execution Model Path-Vector Protocol Example Query specifications protocol implementation Example Queries Realizing the Connection Declarative Overlays Conclusion

23. Example Routing Queries Best-Path Routing Distance Vector Dynamic Source Routing Policy Decisions QoS-based Routing Link-state Multicast Overlays (Single-Source & CBT) Compact, natural representation Customization: easy to make modifications to get new protocols Connection between query optimization and protocols Takeaways:

24. R1: path(@S,D,,C) link(@S,D,C) R2: path(@S,D,,C) C=C 1 +C 2, Query: path(@S,D,,C) link(@S,Z,C 1 ), path(Z,D,,C 2 ), All-pairs All-paths, P=(S,D). P P P2P2 P=S P 2. P

25. R1: path(@S,D,P,C) link(@S,D,C), P=(S,D). R2: path(@S,D,P,C) link(@S,Z,C 1 ), path(@Z,D,P 2,C 2 ), C=C 1 +C 2, Query: bestPath(@S,D,P,C) R3: bestPathCost(@S,D,min ) path(@S,D,Z,C). R4: bestPath(@S,D,Z,C) bestPathCost(@S,D,C), path(@S,D,P,C). All-pairs Best-path P=S P 2.

26. R1: path(@S,D,P,C) link(@S,D,C), P=(S,D). R2: path(@S,D,P,C) link(@S,Z,C 1 ), path(@Z,D,P 2,C 2 ), C=FN(C 1,C 2 ), Query: bestPath(@S,D,P,C) R3: bestPathCost(@S,D,AGG ) path(@S,D,Z,C). R4: bestPath(@S,D,Z,C) bestPathCost(@S,D,C), path(@S,D,P,C). Customizable Best-Paths Customizing C, AGG and FN: lowest RTT, lowest loss rate, highest capacity, best-k P=S P 2.

27. All-pairs All-paths R1: path(@S,D,,C) link(@S,D,C) R2: path(@S,D,,C) C=C 1 +C 2, Query: path(@S,D,,C) link(@S,Z,C 1 ), path(@Z,D,,C 2 ),, P=(S,D). P P P2P2 P=S P 2. P

28. R1: path(@S,D,,C) link(@S,D,C). R2: path(@S,D,,C) link(@S,Z,C 1 ), path(@Z,D,,C 2 ), C=C 1 +C 2 Query: (@S,D,,C) Distance Vector D ZW R3: shortestLength(@S,D,min ) path(@S,D,Z,C). R4: nextHop(@S,D,Z,C) nextHop(@S,D,Z,C), shortestLength(@S,D,C). ZnextHop Count to Infinity problem?

29. R1: path(@S,D,D,C) link(@S,D,C) R2: path(@S,D,Z,C) link(@S,Z,C 1 ), path(@Z,D,W,C 2 ), C=C 1 +C 2 R3: shortestLength(@S,D,min ) path(@S,D,Z,C). R4: nextHop(@S,D,Z,C) nextHop(@S,D,Z,C), shortestLength(@S,D,C). Query: nextHop(@S,D,Z,C) Distance Vector with Split Horizon, W!=S

30. R1: path(@S,D,D,C) link(@S,D,C) R2: path(@S,D,Z,C) link(@S,Z,C 1 ), path(@Z,D,W,C 2 ), C=C 1 +C 2, W!=S R4: shortestLength(@S,D,min ) path(@S,D,Z,C). R5: nextHop(@S,D,Z,C) nextHop(@S,D,Z,C), shortestLength(@S,D,C). Query: nextHop(@S,D,Z,C) Distance Vector with Poisoned Reverse R3: path(@S,D,Z,C) link(@S,Z,C 1 ), path(@Z,D,W,C 2 ), C=, W=S

31. All-pairs All-Paths R1: path(@S,D,P,C) link(@S,D,C), P= (S,D). R2: path(@S,D,P,C) C=C 1 +C 2, Query: path(@S,D,P,C) link(@S,Z,C 1 ), path(@Z,D,P 2,C 2 ), P=S P 2.

32. Dynamic Source Routing Predicate reordering: path vector protocol dynamic source routing R1: path(@S,D,P,C) link(@S,D,C), P= (S,D). R2: path(@S,D,P,C) C=C 1 +C 2, Query: path(@S,D,P,C) path(@S,Z,P 1,C 1 ), link(@Z,D,C 2 ), P=S P 2. P=P 1 D.

33. Other Routing Examples Best-Path Routing Distance Vector Dynamic Source Routing Policy Decisions QoS-based Routing Link-state Multicast Overlays (Single-Source & CBT)

34. Outline Background The Connection: Routing as a Query Realizing the Connection Dataflow Generation and Execution Recursive Query Processing Optimizations Semantics in a dynamic network Beyond routing: Declarative Overlays Conclusion

35. Dataflow Graph Nodes in dataflow graph (elements): Network elements (send/recv, cc, retry, rate limitation) Flow elements (mux, demux, queues) Relational operators (selects, projects, joins, aggregates) Strands Messages Network In Messages Network Out Single P2 Node

36. Dataflow Strand Input Tuples Output Tuples Element 1 Element 2 Element n Input: Incoming network messages, local table changes, local timer events … Strand Elements Condition: Process input tuple using strand elements Output: Outgoing network messages, local table updates

37. Rule Dataflow Strands R2: path(@S,D,P) link(@S,Z), path(@Z,D,P 2 ), P=S P2.

38. Localization Rewrite Rules may have body predicates at different locations: R2: path(@S,D,P) link(@S,Z), path(@Z,D,P 2 ), P=S P 2. R2b: path(@S,D,P) linkD(S,@Z), path(@Z,D,P 2 ), P=S P 2. R2a: linkD(S,@D) link(@S,D) Matching variable Z = Join Rewritten rules: Matching variable Z = Join

39. Dataflow Strand Generation Strand Elements path Join path.Z = linkD.Z linkD Project path(S,D,P) Send to path.S R2b: path(@S,D,P) linkD(S,@Z), path(@Z,D,P 2 ), P=S P 2. Network In linkD Join linkD.Z = path.Z path Project path(S,D,P) Send to path.S

40. Recursive Query Evaluation Semi-naïve evaluation: Iterations (rounds) of synchronous computation Results from iteration i th used in (i+1) th Path Table 8 7 3-hop 10 92 1 1-hop 36 5 2-hop 4 Link TableNetwork 5 10 0 2 1 3 4 6 8 7 Problem: Unpredictable delays and failures 9

41. Pipelined Semi-naïve (PSN) Fully-asynchronous evaluation: Computed tuples in any iteration pipelined to next iteration Natural for distributed dataflows Path Table 4 1 7 Link TableNetwork 2 5 8 3 6 9 10 5 0 2 1 3 4 6 8 7 9 Relaxation of semi-naïve

42. Pipelined Evaluation Challenges: Does PSN produce the correct answer? Is PSN bandwidth efficient? I.e. does it make the minimum number of inferences? Duplicate avoidance: local timestamps Theorems: RS SN (p) = RS PSN (p), where RS is results set No repeated inferences in computing RS PSN (p) p(x,z) :- p 1 (x,y), p 2 (y,z), …, p n (y,z), q(z,w) recursive w.r.t. p

43. Outline Background The Connection: Routing as a Query P2 Declarative Networking System Dataflow Generation and Execution Recursive Query Processing Optimizations Beyond routing: Declarative Overlays Conclusion

44. Overview of Optimizations Traditional: evaluate in the NW context Aggregate Selections Magic Sets rewrite Predicate Reordering New: motivated by NW context Multi-query optimizations: Query Results caching Opportunistic message sharing Cost-based optimizations (work-in-progress) Neighborhood density function Hybrid rewrites PV/DV DSR Zone Routing Protocol

45. Aggregate Selections Prune communication using running state of monotonic aggregate Avoid sending tuples that do not affect value of agg E.g., shortest-paths query Challenge in distributed setting: Out-of-order (in terms of monotonic aggregate) arrival of tuples Solution: Periodic aggregate selections Buffer up tuples, periodically send best-agg tuples

46. Aggregate Selections Evaluation P2 implementation of routing protocols on Emulab (100 nodes) All-pairs best-path queries (with aggregate selections) Aggregate Selections reduces communication overhead More effective when link metric correlated with network delay Periodic AS reduces communication overhead further

47. Outline Background The Connection: Routing as a Query Realizing the Connection P2: Declarative Routing Engine Beyond routing: Declarative Overlays Conclusion

48. Declarative Router Recall: Declarative Routing P2 Engine Declarative Queries Control Plane Forwarding Plane Neighbor Table updates Forwarding Table updates Packets

49. Declarative Overlays Declarative Queries Application level Internet Declarative Overlay Node Packets P2 Engine Default Internet Routing Control and forwarding Plane

50. Declarative Overlays More challenging to specify: Not just querying for routes using input links Rules for generating overlay topology Message delivery, acknowledgements, failure detection, timeouts, periodic probes, etc… Extensive use of timer-based event predicates: ping(@D,S) :- periodic(@S,10), link(@S,D)

51. P2-Chord Chord Routing, including: Multiple successors Stabilization Optimized finger maintenance Failure detection 47 rules 13 table definitions MIT-Chord: x100 more code Another example: Narada mesh in 16 rules 10 pt font

52. Actual Chord Lookup Dataflow

53. P2-Chord Evaluation P2 nodes running Chord on 100 Emulab nodes: Logarithmic lookup hop-count and state (correct) Median lookup latency: 1-1.5s BW-efficient: 300 bytes/s/node

54. Moving up the stack Querying the overlay: Routing tables are views to be queried Queries on route resilience, network diameter, path length Recursive queries for network discovery: Distributed Gnutella crawler on PlanetLab [IPTPS 03] Distributed web crawler over DHTs on PlanetLab Oct 03 distributed crawl: 100,000 nodes, 20 million files

55. Outline Background The Connection: Routing as a Query Realizing the Connection Beyond routing: Declarative Overlays Conclusion

56. A Sampling of Related Work Databases Recursive queries: software analysis, trust management, distributed systems diagnosis Opportunities : Computational biology, data integration, sensor networks Networking XORP – Extensible Routers High-level routing specifications Meta-Routing, Routing logic

57. Future Directions Declarative Networking: Static checks on desirable network properties Automatic cost-based optimizations Component-based network abstractions Core Internet Infrastructure Declarative specifications of ISP configurations P2 deployment in routers

58. Distributed Data Management on Declarative Networks Run-time cross-layer optimizations: Reoptimize data placement and queries Reconfigure networks based on data and query workloads P2: Declarative Networks Customized routes, DHTs, Flood, Gossip, Multicast Mesh Distributed Algorithms Consensus (Harvard), 2PC, Byzantine, Snapshots (Rice/Intel), Replication P2P Search, network monitoring, P2P data integration, collaborative filtering, content distribution networks… Data Management Applications Distributed Queries SQL, XML, Datalog

59. Other Work Internet-Scale Query Processing PIER – Distributed query processor on DHTs http://pier.cs.berkeley.edu [VLDB 2003, CIDR 2005] http://pier.cs.berkeley.edu P2P Search Infrastructures P2P Web Search and Indexing [IPTPS 2003] Gnutella measurements on PlanetLab [IPTPS 2004] Distributed Gnutella crawler and monitoring Hybrid P2P search [VLDB 2004]

60. Contributions and Summary P2 Declarative Networking System Declarative Routing Engine Extensible routing infrastructure Declarative Overlays Rapid prototyping overlay networks Database fundamentals Query language New distributed query execution strategies and optimizations Semantics in dynamic networks Period of flux in Internet research Declarative Networks can play an important role

61. Thank You