The Structuring of Systems Using Upcalls David D Clark Presented by: Prassnitha Sampath

Background – Layered Architectures Layered architectures de facto standard for designing complex systems, for example – TCP/IP seven layered stack – OSI reference model – 3-tier client-server architecture Layered architectures have inherent advantages – Helps system organization – Acyclic dependency graph, better for verification – Stand alone testing at any layer – Modularity and portability Layered architectures typically have an implicit hierarchy of trust and reliability – As we move down the stack, the “trustworthiness” of the system increases – If a higher layer of the stack fails, the lower levels are still uncorrupted

Limitations of Layered Architectures The data flow model has an inherent bias top to bottom – “Downcalls”: Data flows down the stack through traditional function calls, rather quick and efficient – Data flow upwards is typically inefficient Either inter-process signaling or polling to indicate data availability Some cases require data buffering and/or copying of data from lower layer to a higher layer Proposal to address the inefficiencies – Add support for “Upcalls”; allow procedure calls from lower layers to higher layers. Intended to address the imbalance of the system – Enable “true” subroutine calls both upwards and downwards. Intended to modulate and balance data flow

Proposed System Organization Swift – Operating System built with the proposed paradigm Each layer is composed of a set of sub- routines which can be invoked both upwards and downwards Coordination across subroutines within the same layer using shared memory Multitask Modules Easy access to relevant state information for all tasks User tasks span a vertical stack all the way from the top layers to the bottom layers Upper layer is always “available” to the lower layer and vice versa

An Example Application Layer Transport Layer Network Layer

An Example Initiates a connection and indicates which function needs to be called when data is available Behavior when data is received. Opens a network layer connection, and sets it up to indicate which function to call when data is received on the network layer. Updates data structures to remember which function to callback Transport layer receive function that reformats the input, and redirects to the application layer receive function. Utility function from the transaction layer. This function understands the format of packets and the port they are associated with. Primarily used by network layer to identify port Opens a network layer connection, spawns a new task that listens on the port, and returns control. It also updates some network layer data structures to associate the listener process with the port Services packets as they come. Redirects to the appropriate transaction layer receive function. This will be called by the net-dispatch, which is a common function called by the interrupt handler Services raw network packets as they come. Puts the packet in the data structure corresponding to the port on which the packet was received. Wakes up the thread associated with the port

Advantages of Upcall Methodology – I Efficiency – All communication between layers is through subroutine calls, which makes it efficient – There is a smaller need for Inter Process Communication, which also adds to efficiency Closely written layers which provides scope for optimizations as against a one-size-fits-all approach – E.g. the transport layer can indicate availability of another packet to send and enable piggy-backing of the ack with new data to send

Advantages of Upcall Methodology – II Programmers tend to understand procedure calls more easily than standardized interfaces Data flow control simplicity – Data is never duplicated in the transaction layer – Application layer can itself back-pressure the network layer – Do not need to enable data buffering and back- pressuring between layers – Disadvantage: The above is also a short-coming because a poorly written application can swamp the full network

Disadvantages of Upcall Methodology – I Reliability: If an upper layer fails, then it potentially brings down the full framework Potential Solutions: – Organize the multi-task modules in a fault-resilient manner by dividing up the “private” variables from the “shared” state – “Tasks” should be expendable Permanent damage to the environment such as a lock held, etc, should be avoided by design – Implement layer specific cleanup mechanisms to reclaim lost resources due to defunct tasks

Disadvantages of Upcall Methodology – II Trust: Common/Shared software for the layers is more “trustworthy” due to a formal spec on the behavior Potential Solutions: – Rely on “competent” programmers – Humbly submit, these are a scarce resource

Disadvantages of Upcall Methodology – III Recursive behavior: One very desirable property of the layered architecture is that the dependency graph is acyclic. – This makes testing and verification simple. – Also the lower layer is in a consistent state w.r.t. the upper layers within each function. However, if upcalls are allowed, the system state can be changed by the upcalled procedure. Potential Solutions: – Lower layer reverifies system state after each upcall. – Prohibit recursive downcall. Preserver acyclic nature of function call graph Possibly use flags for down communication – Use a combination of a pull-push mechanism for poster-boy cases of recursive downcalls For eg. Instead of requesting the upper layer to send data triggering a downcall to send data, send structure for the upper layer to fill

Disadvantages of Upcall Methodology – IV Multitask Module: Multitask module requires parallel programming with shared memory, prone to bugs – Inappropriate usage of locks possible Potential Solutions: – Code carefully, like coding any other parallel system. – Use other inter-process communication techniques for synchronization. Possibly have a main task that performs the actual actions Starts resembling traditional layered architecture with the primary task being a layer in itself