Orbited Scaling Bi-directional web applications A presentation by Michael Carter

Classic Web Navigation ● Browser Submits Request to server  GET /project.html HTTP/1.1 ● Server responds  HTTP/ OK  Content-length: 25  Hello! This is my project.

Ajax ● XMLHttpRequest  Behind the scenes  Javascript  Asynchronous ● Allows for incremental updates without navigation

Problem ● HTTP Pull model up until now ● What about the other direction? ● No server initiated transfers allowed ● Consider a chat application

A First Solution: Polling ● Browser makes Ajax requests every 0.5 seconds ● Server returns all events since last request ● Simulates bi-directional flow ● Allows us to make a chat application

Polling Problems ● High Latency  500 ms immediate overhead  Plus latency between the server and browser  And latency from the event originator and the server ● Heavy Load  2/req sec per user  1000 users = 2000 req / sec  Well beyond the capacity of most dynamic web applications

A Better Solution: Comet ● Browser makes a request for an event ● The Server accepts the request but does not respond ● Later, when an event occurs, the server sends data via the original request ● The Browser processes the data, then waits for more data

Comet Advantages ● Eliminates Latency overhead of polling ● Allows for true bi-directional flow ● Not difficult to implement in javascript

Comet Problems ● Web Servers can't handle comet + concurrency  Requests are expected to be short lived  Most servers are thread based  Each request goes to its own thread  So generally 1000 users can be handled with just threads.  Now connections are long lasting and sit idle  So threads can handle users

Comet Problems ● Scaling  Comet is relatively straightforward if you have a single server node  But if your capacity exceeds one node, you must add another  Classically, server nodes can be treated identically  But now its unclear which user is connected to which server, waiting for events

Comet Problems ● Integration  Difficult to add comet to your applications  Must access low-level web server apis  Must keep track of state somehow  Most web servers make this clumsy at best, and sometimes nearly impossible

Orbited: Solving Comet's problems ● Orbited is  separate, runs alongside the web server  Event-based. It can handle thousands of concurrent requests due to the lack of threads.  Simplistic. Keeps track of minimal state  Scalable. Due to the simplicity it is easy to scale Orbited nodes

A Round Trip (an Orbit) ● Our Browser asks Orbited for an event ● Orbited accepts the requests but doesn't reply ● Browser asks the server to encode some music ● The server responds with 'Ok. Will notify you later.' ● The server finishes its tasks ● The server connects to Orbited and tells it to respond to the user ● Orbited answers the request from long ago: 'All done!'

Orbited: Chat app ● Two functions  Join 'User Michael has joined'  Msg 'Michael: hello'

Orbited Client API ● To send a msg to a user, simply call  orbit.event(user, session, location, msg) ● Or to send a msg to multiple users call  orbit.event(recipients, msg)

Scaling Orbited ● Client uses a hash function to map users to nodes ● Javascript uses the same hash function ● The nodes are oblivious – they share no state ● To scale, simply add more nodes and update the hash function ● Scales Linearly

Orbited Caveats ● Due to Browser security, comet pages and content pages must have the same host name originator. ● Therefore you must do complex DNS server manipulations ● Orbited includes a development mode where it acts as a proxy to the web application server so you can avoid the DNS configuration ● Some firewalls shutdown idle HTTP connections after 60 seconds. So a new connection must constantly be remade.

Further Research ● Additional transports ● Triangular Routing  Allows dynamic load balancing  Eliminates Proxy overhead  No need for complex dns configuration ● More complete javascript support

Conclusion ● Orbited is fast ● Orbited is scalable ● Orbited is easy to integrate ● Orbited is a very feasible solution to the comet problem