1. Scalability: Load Balancing
From one to many application servers

2. Scalability [Bass et al., 2012]
Vertical Scalability (Scale up)
Adding more resources to a physical unit
More RAM, more Disk, more CPU
Horizontal Scalability (Scale out)
Adding more resources to logical units
More servers
In cloud computing Elasticity
Add/remove VMs to resource pool
Henrik Bærbak Christensen

3. Henrik Bærbak Christensen
Discussion
The quality attribute scalability was not mentioned in 2nd edition of Bass et al. [2003]
Basically because ‘scalability’ is just a sub quality of a more abstract one…
Performance
Why do we need to scale? Because we need more performance!
Henrik Bærbak Christensen

4. Architectural Evolution
SkyCave, at course start
Henrik Bærbak Christensen

5. Architectural Evolution
SkyCave, more or less now
Henrik Bærbak Christensen

6. Henrik Bærbak Christensen
Issues…
There are some severe scaling issues
Ex: Some users on SkyCave
One user ~ one request every two seconds
That is: requests per second
Remember: accept socket connection, read JSON, execute (potential several DB accesses) and compute reply JSON, write JSON, close socket
Worse:
If 100 issue ‘weather’ and we hit our weather service 8 sec timeout?
Basically a performance challenge !
Henrik Bærbak Christensen

7. Henrik Bærbak Christensen
Technical Issues
We need to address the following issues
From one to many servers Horizontal Scaling
Load Balancing
Distribute requests over a set of servers, not just one
Session Management
If Magnus logs in on ‘server1’ but next request hits ‘server2’
The ‘player cache’ is cold! The system gets a cold and coughs
From one to many threads
Each server is single threaded
One weather time-out will stall all other incoming requests !
Henrik Bærbak Christensen

8. Henrik Bærbak Christensen
Load Balancing
Henrik Bærbak Christensen

9. Henrik Bærbak Christensen
Load Balancer
Load Balancer
Makes the pool of servers under the load balancer appear as a single server with high computing capacity. [Bahga et al., 2014]
Basically a ProxyServer that distribute requests to a pool of servers…
Henrik Bærbak Christensen

10. Henrik Bærbak Christensen
Examples
HA Proxy
TCP/HTTP proxy
Used by github, airbnb, instagram, stack overflow, …
Nginx
Round-robin DNS
Multiple ip addresses associated with single domain
Hardware load balancers
Messaging systems
Can load balance… and a lot more…
Henrik Bærbak Christensen

11. Statefull versus Stateless
Session Management
Statefull versus Stateless

12. Henrik Bærbak Christensen
Servers and State
The main requirement on any server that is load balanced:
It must be stateless
Stateful:
Has cached/stored state about given session with a client in the server
Stateless
No stored state about given session
Henrik Bærbak Christensen

13. Henrik Bærbak Christensen
Sessions?
Some domains do not require sessions
Simple web browsing
Simple data storage
Others domains, sessions are vital
Shopping basket while web shopping
Game interaction
SkyCave
Henrik Bærbak Christensen

14. Architectural Design
Ok, so SkyCave has already encapsulated session management?
Encapsulate what varies
Program to an interface
Favor object composition
Henrik Bærbak Christensen

15. Henrik Bærbak Christensen
Session Handling
Bahga et al., 2014
Sticky sessions
All requests from session ‘a’ are routed to same server
Session database
Session data is stored in session database = persistent storage shared by all servers
Browser cookies (client session)
Session data is stored in client, sent together with each request
URL re-writing
Henrik Bærbak Christensen

16. Henrik Bærbak Christensen
Discussion
What kind of session management is used in SkyCave at the moment?
Sticky session, session database, client sessions
Could we code a ‘client session’ approach?
What would it require?
And… Why can we not use the approach  ?
Hint: Think about dual login from different clients.
Henrik Bærbak Christensen

17. Threading in Servers
More next week 

18. Henrik Bærbak Christensen
Summary
More users means more resource demand
Answer: Add resources (or become more efficient)
Horizontal or Vertical
Load Balancing:
Make a server cluster appear like one server
Session Management
Handle that different servers are ‘hit’ by given client
Henrik Bærbak Christensen