1NinjaScript Presentation CS264December 2, 1998 NinjaScript A Dataflow Language for Composing Network Services in Ninja Andrew Begel

2NinjaScript Presentation CS264December 2, 1998 Overview What is Ninja? What’s the problem? How can NinjaScript help? NinjaScript implementation Future Work

3NinjaScript Presentation CS264December 2, 1998 What is Ninja? A distributed systems architecture created by Brewer, Joseph, Katz, Kubi, and lots of graduate students Fault-tolerant, highly-available services exist in the network ether (takes ideas from HotBot) Currently based on Java with JavaRMI for communications.

4NinjaScript Presentation CS264December 2, 1998 What is the problem? We want to create applications that use many services. Existing systems like CORBA, DCOM, ILU, or JavaRMI provide client-server connections between pairs of nodes. That’s not enough!

5NinjaScript Presentation CS264December 2, 1998 Pathways are the answer. We want to develop compositions of multiple services. Individual connections have no knowledge of the bigger picture. Benefits from the path metaphor: –Can ask questions about the path: congestion, bottlenecks, bandwidth, latency, admission control, security –Language Benefits: Optimizations (pipelining, commutative services), Debugging

6NinjaScript Presentation CS264December 2, 1998 NinjaScript It’s a path-based programming language! Think of a big graph with a start node and some finish nodes. Several language metaphors wrapped up in one. –Lambda Calculus –Multiple, Conditional Outputs –Late Binding –Subroutines –Soft State –Type Safety

7NinjaScript Presentation CS264December 2, 1998 An Example Program Voice-Activated Light Switch The basic path. (To all other nodes, the interpreter is a black box.)

8NinjaScript Presentation CS264December 2, 1998 The Interpreter The internals of the interpreter. It has a subroutine call to the base station and ultimately outputs a command to the light switch.

9NinjaScript Presentation CS264December 2, 1998 Implementation Path program is description of nodes and edges of a graph. Nodes will run services that implement published interfaces. Edges have properties such as reliability, in- order/out-of-order transmission, buffering, flow control.

10NinjaScript Presentation CS264December 2, 1998 Path Threads A path thread consists of a 4-tuple –unique ID –graph description: nodes (service name, bootstrap code) edges (source, sink, properties) start node and finish nodes –current state of execution (state, IP address, method signature) –data payload (function arguments)

11NinjaScript Presentation CS264December 2, 1998 Node Operation Upon receiving a packet, check the unique ID. If it’s new, create a new thread, else restore the old thread to respond to the data. Unpack the packet data, and check the payload for function arguments. Unpack these and store in the thread. Invoke the method identified by the current state.

12NinjaScript Presentation CS264December 2, 1998 Node Operation (2) After the method returns its arguments, package them up in a new data payload. Consult the graph description to determine the next state. If this is a finish node, exit. Otherwise, package up a new packet and send it to the next nodes.

13NinjaScript Presentation CS264December 2, 1998 Extra Features (not done yet) Subroutines Conditional Output –Enables more specific control over which outputs go where, if they go anywhere at all. Soft State –Path local –Node global

14NinjaScript Presentation CS264December 2, 1998 Future Work Make the whole thing work. Incorporate multi-language service support in the compiler backend. Create alternate graphical front-end. Integrate existing transport layers (IIOP, RMI) and alternate transport protocols. Develop new node code format (currently using Solaris Sparc shared libraries). Develop path-based optimization, debugging and diagnostic tools. Support paths as first-class objects (exception handling).