1. Distributed Data Flow Language for Multi-Party Protocols Krzysztof Ostrowski †, Ken Birman †, Danny Dolev § † Cornell University, § Hebrew University {krzys|ken}@cs.cornell.edu, dolev@cs.huji.ac.il

2. Introduction

3. http://liveobjects.cs.cornell.edu

4. distributed peer-to-peer protocols

5. http://liveobjects.cs.cornell.edu distributed peer-to-peer protocols multicast locking replication commit

6. http://liveobjects.cs.cornell.edu distributed peer-to-peer protocols multicast locking replication commit shared content

7. http://liveobjects.cs.cornell.edu distributed peer-to-peer protocols multicast locking replication commit replicated content

8. http://liveobjects.cs.cornell.edu multicast locking replication commit replicated content distributed peer-to-peer protocols synchronization, coordination

9. http://liveobjects.cs.cornell.edu

13. basic building block

14. http://liveobjects.cs.cornell.edu basic building block

15. http://liveobjects.cs.cornell.edu basic building block

16. http://liveobjects.cs.cornell.edu need lots of different objects (protocols)

17. http://liveobjects.cs.cornell.edu need lots of different objects (protocols) user-defined objects

18. How to Implement New Objects? custom user-defined object

19. custom user-defined object Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. How to Implement New Objects?

20. custom user-defined object Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. How to Implement New Objects?

21. custom user-defined object Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. How to Implement New Objects?

22. higher-level logic (making decisions) Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. lower-level logic (e.g., ACKs, timeouts, building rings/trees, internal bookkeeping) intermingled, tightly-coupled How to Implement New Objects?

23. higher-level logic (making decisions) lower-level logic (e.g., ACKs, timeouts, building rings/trees, internal bookkeeping) intermingled, tightly-coupled How to Implement New Objects?

24. higher-level logic (making decisions) lower-level logic (e.g., ACKs, timeouts, building rings/trees, internal bookkeeping) intermingled, tightly-coupled How to Implement New Objects?

25. less flexibility harder to write/debug Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. error- prone less sophisticated How to Implement New Objects?

26. less flexibility harder to write/debug Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. error- prone less sophisticated How to Implement New Objects?

27. less flexibility harder to write/debug Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. error- prone less sophisticated How to Implement New Objects?

28. less flexibility harder to write/debug Java / C# / C++ protocol composition Frameworks (Ensemble, BAST, Appia) MACE, P2, etc. error- prone less sophisticated How to Implement New Objects?

29. lower-level logic (e.g., ACKs, timeouts, building rings/trees, internal bookkeeping) higher-level logic (making decisions) separation of concerns

30. How to Implement New Objects? separation of concerns programmer compiler and runtime

31. How to Implement New Objects? separation of concerns programmer compiler and runtime

32. Our Approach: Flows Objects Aggregation Batching Recursion

66. e1e1

67. e2e2

68. x 1 crashed

69. e3e3

70. x 4 joined

71. e4e4

72. e5e5

73. e6e6

74. example: distributed locking protocol

75. application layer

76. example: distributed locking protocol application layer protocol layer

77. example: distributed locking protocol application layer protocol layer

78. example: distributed locking protocol

97. Our Approach: Flows Objects Aggregation Batching Recursion

98. example: distributed locking protocol

115. Our Approach: Flows Objects Aggregation Batching Recursion

116. input flow output flow aggregation

117. input flow output flow aggregation output event

118. input flow output flow output event input events

121. example: leader election protocol 3 3 9 6 aggregation with “min”

122. example: leader election protocol aggregation with “min” 3 9 6 3

123. example: leader election protocol aggregation with “min” 3 9 6 3

124. example: leader election protocol aggregation with “min” 3 9 6 3

125. Different flavors of aggregation: 1.In-order, 2.Guarded, 3.Coordinated, 4.Etc. formal, abstract properties can reason about global behavior Separation of Concerns

126. We don’t specify: 1.How values are aggregated (in the network, in a centralized fashion) 2.Which nodes interact with one-another 3.What protocol is used (token ring, scalable tree, gossip) 4.Where the aggregated values emerge 5.When and how often aggregation occurs

127. Separation of Concerns We don’t specify: 1.How values are aggregated (in the network, in a centralized fashion) 2.Which nodes interact with one-another 3.What protocol is used (token ring, scalable tree, gossip) 4.Where the aggregated values emerge 5.When and how often aggregation occurs

128. Separation of Concerns We don’t specify: 1.How values are aggregated (in the network, in a centralized fashion) 2.Which nodes interact with one-another 3.What protocol is used (token ring, scalable tree, gossip) 4.Where the aggregated values emerge 5.When and how often aggregation occurs

129. Separation of Concerns We don’t specify: 1.How values are aggregated (in the network, in a centralized fashion) 2.Which nodes interact with one-another 3.What protocol is used (token ring, scalable tree, gossip) 4.Where the aggregated values emerge 5.When and how often aggregation occurs

130. Separation of Concerns We don’t specify: 1.How values are aggregated (in the network, in a centralized fashion) 2.Which nodes interact with one-another 3.What protocol is used (token ring, scalable tree, gossip) 4.Where the aggregated values emerge 5.When and how often aggregation occurs

131. Separation of Concerns We don’t specify: 1.How values are aggregated (in the network, in a centralized fashion) 2.Which nodes interact with one-another 3.What protocol is used (token ring, scalable tree, gossip) 4.Where the aggregated values emerge 5.When and how often aggregation occurs

132. Aggregation with a Token Ring

133. {1..3,5..6}{2..4, 6}{1..6}

134. Aggregation with a Token Ring {1..3,5..6}{2..4, 6}{1..6} token leader node

135. Aggregation with a Token Ring {1..3,5..6}{2..4, 6}{1..6} token

136. Aggregation with a Token Ring {1..3,5..6} {2..4, 6}{1..6}

137. Aggregation with a Token Ring {1..3,5..6} {2..4, 6}{1..6}

138. Aggregation with a Token Ring {1..3,5..6} {2..4, 6} {1..6}  intersect

139. Aggregation with a Token Ring {2..3, 6} {1..6}

140. Aggregation with a Token Ring {2..3, 6} {1..6}

141. Aggregation with a Token Ring {2..3, 6} {1..6}  intersect

142. Aggregation with a Token Ring {2..3, 6}

143. Our Approach: Flows Objects Aggregation Batching Recursion

144. Batched Processing with Sets node x node y node z

145. Batched Processing with Sets node x node y node z

146. Batched Processing with Sets node x node y node z

147. Batched Processing with Sets node x node y node z 123456 123456 123456 7 7 7

148. Batched Processing with Sets node x node y node z 12356 47 2364 157 123456 7 DROPPED:

149. Batched Processing with Sets node x node y node z 123562364123456

150. Batched Processing with Sets node x node y node z 123562364123456

151. Batched Processing with Sets node x node y node z 123562364123456 Rec({1..3,5..6})

152. Batched Processing with Sets node x node y node z 123562364123456 Rec({1..3,5..6}) Rec({2..4, 6})

153. Batched Processing with Sets node x node y node z 123562364123456 Rec({1..3,5..6})Rec({2..4, 6}) Rec({1.. 6})

154. Batched Processing with Sets node x node y node z 123562364123456 Rec({1..3,5..6})Rec({2..4, 6}) Rec({1.. 6})

155. example: which packets are stable? {2..3,6} {1..3, 5..6} {1..6} aggregation with “  ” {2..4,6}

156. example: which packets are stable? {2..3,6} {1..3, 5..6} {1..6} aggregation with “  ” {2..4,6}

157. example: which packets are stable? {2..3,6} {1..3, 5..6} {1..6} aggregation with “  ” {2..4,6}

158. example: which packets are stable? {2..3,6} {1..3, 5..6} {1..6} aggregation with “  ” {2..4,6}

159. Our Approach: Flows Objects Aggregation Batching Recursion

160. Examples

161. 01: object lock ( bool wants ) : bool holds 02: { 03: same int owner; 04: where ( wants ) 05: owner := elect ( id ); 06: holds := wants  ( owner = id ); 07: } embedding distributed locking protocol

162. 01: object elect ( up int candidate ) 02: : s-up int leader 03: { 04: s-up int elected := 0; 05: where ( fresh elected 06:  elected ≤ candidate ) 07: elected := min candidate; 08: leader := elected; 09: } leader election protocol the core part

163. Conclusions

164. 1.Collaboration requires custom protocols a)Fine-tuned for a particular application semantics b)Fine-tuned for different networks and workloads 2.Existing protocol languages don’t suffice a)Developers need a clean separation of concerns 3.Modeling protocols as distributed flows a)Captures high-level protocol semantics concisely b)Reduces a coding burden on protocol developers c)Supports reasoning about the protocol behavior d)Provides a high degree of architectural flexibility Conclusions

165. Thanks