1. A Throttling Layer-7 Web Switch James Furness

2. Motivation & Goals Specification & Design Design detail Demonstration Conclusion

3. 3A Throttling Layer-7 Web Switch Project Goals Present a heterogeneous pool of web servers as a single virtual server Maximise availability and response times by throttling Balance load between servers in pool Design as a modular framework to maximise extensibility

4. 4A Throttling Layer-7 Web Switch Motivation Rapid growth in usage of the world-wide web Rapid growth in bandwidth available to end user Users expect timely responses Source: Netcraft

5. 5A Throttling Layer-7 Web Switch Motivation Web sites must continue to provide timely responses to all users under high demand Options: –Slow incoming request rate –Increase capacity/parallelism –Increase throughput

6. Motivation & Goals Specification & Design Design detail Demonstration Conclusion

7. 7A Throttling Layer-7 Web Switch System Specification Cluster-based Web system Layer-7 switching Virtual File System allows multiple versions of content to be provided on multiple backend servers Aim: Degrade responses from highest quality as load increases

8. 8A Throttling Layer-7 Web Switch Control Layer Architecture VFS performs virtual to physical mapping and stores Path monitoring data Policy engine prioritises alternative versions of content according to load Control Layer interface provides façade System monitors and dispatching algorithm user-specified Control Layer Interface Control Layer Routing Layer Virtual Path Mapper Dispatching Algorithm Configuration Module System Monitors Callbacks Policy Engine Virtual File System Monitoring Data

9. 9A Throttling Layer-7 Web Switch System Architecture Two-layer approach Separate routing and the logic making routing decisions Abstracts routing method from Control Layer Control Layer can be retro- fitted to any web switch Routing Layer is a minimally modified proxy server Control Layer Routing Layer 1. Request 2. Virtual Path 3. Physical Paths 4. Ordered Physical Paths 5. Rewritten Request 6. Response 7. Response Web Switch User Decode Request Virtual to Physical Translation Load Balancing Rewrite Request Pool Servers Forward Response

10. 10A Throttling Layer-7 Web Switch System Architecture Scalable Content-blind load balancer (Layer-4) Web Switches (Layer-7) Server pool Portable (Java)

11. Motivation & Goals Specification & Design Design detail Demonstration Conclusion

12. 12A Throttling Layer-7 Web Switch Default Policy Carry alternative pages of varying bandwidth/load Switch between versions to prevent flooding of outbound bandwidth and to reduce response times of dynamic pages under high load Prioritise physical paths using: –Monitoring data System data Host data Path data –Metadata Load Weighting Bandwidth Weighting

13. 13A Throttling Layer-7 Web Switch Default Monitoring Data 1.Collect system bandwidth usage 2.Collect physical path response times Response time provides a platform independent measure of end-to-end system performance Calibrate system by including target response time average + standard deviation in metadata Normalise monitored response times using metadata Collect average of recent normalised response times for path and also for all paths on host using sliding window

14. 14A Throttling Layer-7 Web Switch Default Dispatching Algorithm Throttle bandwidth back as connection maximum is approached Favour paths averaging < 1 s.d Reward/punish dynamic pages more than static pages

15. 15A Throttling Layer-7 Web Switch Default Dispatching Algorithm Secondary criteria: Favour hosts with low average standardised response time Favour hosts with least active connections

16. 16A Throttling Layer-7 Web Switch Experimental Results

17. 17A Throttling Layer-7 Web Switch Experimental Results

18. Motivation & Goals Specification & Design Design detail Demonstration Conclusion

19. Motivation & Goals Specification & Design Design detail Demonstration Conclusion

20. 20A Throttling Layer-7 Web Switch Conclusion System designed & implemented conforming to goals Scalable Testing proves an increase in throughput compared to a system without throttling

21. Questions?