Ensemble Fault-Tolerance Security Adaptation

The Horus and Ensemble Projects Accomplishments and Limitations Kent Birman, Bob Constable, Mayk Hayden, Jason Hickey, Christoph Kreitz, Robbert van Renesse, Ohad Rodeh and Werner Vogels Cornell University

Isis, Mother of Horus and Ensemble Toolset for Implementing Virtual Synchronization in a Process Group NYSE Overhead Displays and Tickers Swiss Exchange Communications Architecture VLSI Fabrication Plants (AMD, Siemens, TI) AEGIS Naval Radar and Communication System

Virtually Synchronous Process Group Computing Automatically Managed Membership (adding / removing member, notifying the group of these changes) Multicasts with Ordered Message Delivery Group message passing is coordinated with group membership updates. Ability to transfer the group’s current state to new members.

Virtual Synchrony Model

Isis Pros and Cons Good support for data replication and Robust Dynamic load- balancing server clusters High-Availability  Monolithic, inflexible system. Required a lot of programming to retool it or customize it. Slow Buggy

“to support group communication using a single generic architectural framework within which the basic group communication interfaces are treated separately from their implementation”

Micro-Protocol Stacking a customizable protocol stack for distributed applications. Each process group can share the same application-specific protocol. Different process groups in the same cluster can use different, appropriate protocols.

Handling Message-Loss with Micro-Protocol Stacking (an Example) A message-handling micro-protocol keeps a copy of each outgoing message, and enforces order in incoming messages. An incoming message in the wrong order causes the protocol to send a NAK message to the source, asking for a retransmission of the missed message. When an outgoing message has been determined to be received, it can be removed from storage. Some situations may not need this functionality. In such instances Horus can dynamically leave the message- handling protocol out of the stack to boost speed. Other examples: authentication, encryption layers.

Horus Technologies and Successes Horus Common Protocol Interface Header optimizations and compressions Horus Protocol Accelerator Demonstrated that time-critical, fault- tolerant computing with clusters was feasible.

Limitations of Horus Written in C, no inherent type-checking in language (a problem with protocol stacking where we are passing data from one layer to another) Passing header from layer to layer also requires time; conversions and header extensions may need to occur at each layer. Some protocols may perform redundant operations. New protocol stacks have to be hand-coded and hand- verified for efficiency in C. This requires a commitment of time from the protocol stack designer which can still lead to buggy, inefficient code.

Ensemble “Horus Written in a Formal Language” Ensemble was primarily done to overcome limitations of procedural C used in Horus OCAML--a compiled, object-oriented language syntactically based on Lambda calculus. OCAML is very formal and precise. Automated theorem provers (such as nuPRL) can process formal languages like OCAML easily.

Representing the Protocol Stack in Ensemble Represent the entire protocol stack as a series of nested functions on x: Outgoing x: f 0 (f 1 (f 2 (…f n (x)))) Incoming x: f n (f n-1 (f n-2 (…f 0 (x)))) Conditional Evaluation: If (is_a_msg(x)) f (x) else f(x)

2 Types of Formal Optimizations in Ensemble: Static or layer optimization: Apply logical simplification and truth-condition evaluation (assuming the most common cases) to create a highly efficient compiled code for each micro- layer. Applied once when a protocol is first written to produce individual layer optimization theorems. Dynamic or stack optimization: Compose individual layer optimization theorems into a unified stack optimization theorem, assuming the most common case for each layer. Must be performed for each new protocol stack an application developer wants to deploy.

Designing and Deploying New Protocol Stacks: 1.Choose from a set of micro-protocols (written in layer-optimized OCAML byte code) to compose a new protocol stack. 2.Use NuPRL to optimize the stacked OCAML expressions. 3.Compile the OCAML byte code into machine code and plug it in.

Ensemble Programs Ensembled – the Ensemble Daemon Groupd – the Group Daemon