Cyclone Server Architecture Streamlining Delivery of Popular Content Stanislav Rost John Byers Azer Bestavros

DEALING WITH POPULARITY  Minimize resource consumption necessary for file delivery  In-server caching/prefetching: “cache popular content”  Out-of-server caching/replication: proxies, Content Delivery Networks  Reliable multicast: near-optimal

CONSEQUENCES OF POPULARITY  High degree of concurrency Per-file concurrency  Inefficiency arises from separation of state “Service threads” delivering the same file to multiple clients could share state Server application and the kernel subsystems could share state

OBJECTIVE  Our solution: (Almost) perfect sharing of state among service threads interested in same content (Almost) stateless network stack  Two statements of the same goal: eliminate the marginal cost of serving the same file to additional clients reduce the cost of serving a massive number of requests for the same content to a constant

CACHE MEMORY: CONSTRAINTS AND CONSEQUENCES  Typically, size of a working set far exceeds the capacity of cache memory  Methods of delivery of uncached files are particularly inefficient. Data not cached cannot be shared  Additional objective: cache compactness Make the performance benefits of caching/sharing available to more files

CACHE COMPACTNESS  Conventional method: “sliding buffer” Not efficient: opening the file impacts start-up latency, frequent I/O blocking Not scalable: memory usage grows with clients Difficult to share: asynchronous requests, heterogeneous rates  service threads need data from different positions in file

SLIDING BUFFER SHARING ILLUSTRATED File blocks on disk Clients Sliding buffer ?

IDEAL SOLUTION Objectives: –(Almost) perfect sharing of state among service threads interested in same content –(Almost) stateless network stack –Compact caching with conventional caching-like performance benefits  All can be achieved if the sliding buffer is globally useful. Contents of the buffer always contribute to the transfer progress of all clients downloading the file whose data is in the buffer

TORNADO CODES Original file, n blocks Tornado encoding, k·n blocks Receiver can reconstruct the original file from any n+e distinct blocks e~0.03n linear decoding time ENCODE TRANSMIT/DECODE

DESIGN OVERVIEW The Tornado encoding of the file is stored on disk  A single sliding buffer needed to serve a particular file to any group of clients: Traverses the encoding of the file at the speed of the fastest client Slower clients reconstruct data from the missed blocks from redundant blocks transmitted later  Global usefulness allows relaxation of the in- order delivery constraint: Delivery of the file can begin immediately from the current position of the sliding buffer No need for transmission/retransmission queues: when need to transmit arises, simply transmit the next unsent block from the sliding buffer

CYCLONE SUBSYSTEM

CYCLONE CLIENT/SERVER ARCHITECTURE