2. (c) 2003 - Lindsay Bradford1 Varying Resource Consumption to achieve Scalable Web Services Lindsay Bradford Centre for Information Technology Innovation

3. (c) 2003 - Lindsay Bradford2 Overview Motivation Approaches to Scalability The Approach Selector Prototype Experiments and Results Ongoing and Future Work

4. (c) 2003 - Lindsay Bradford3 Scalability Matters Users expect “service on demand” from the Internet - Bhatti et.al Dynamic web content:  On the increase – Barford et.al  Much harder to scale than static content – Stading et.al Flash crowds a more common occurrence?  Consider: fully Internet enabled China mainland,  SOAP, WSDL, etc. make programmatic access and automation easier. Allows greater client request traffic.

5. (c) 2003 - Lindsay Bradford4 Scalability - Dynamic Content: Static Content: Bottleneck = Bandwidth Dynamic Content: Bottleneck = CPU Dynamic content caching techniques:  Active Query Caching -- Remote Proxy applets, mobile code caching partial content at proxy server(s).  Data Update Propagation (DUP) -- Local and/or Remote Cached dynamic content fragments re-evaluated once base source data changes.  HTML Macro Processing / WEAVE -- Remote Protocol extension to tag static and dynamic parts of response. Static part can then be cached. Remote Cache server constructs complete response.

6. (c) 2003 - Lindsay Bradford5 Dynamic Content Caching:

7. (c) 2003 - Lindsay Bradford6 The Approach Selector (1): Inspired by ``Multimedia’’ Quality Degradation (dropping to ``user acceptable’’ frames/second under load). Alternative to Dynamic Content Caching. Guiding heuristics:  Pick approach that will respond in human acceptable time frame (< 1 second)  Prefer more costly approach to less costly where possible.  Selector must balance approach generation time against target response time.  Limit scope to “Application Programmer” perspective. No modification of supporting technologies (App Servers, etc). What could developers do right now? What limits exist?

8. (c) 2003 - Lindsay Bradford7 Why One Second? Why Degrading Approaches? HCI lessons ignored on the Web: –Interest in and perceived quality of site is inversely proportional to response speed. Content makeup (text/graphics mix) has little effect. – Bhatti et.al.

9. (c) 2003 - Lindsay Bradford8 The Approach Selector(2):

10. (c) 2003 - Lindsay Bradford9 The Approach Selector(3): Unmodified Apache Tomcat 4.1.18 (75 Threads) Approach Selector implemented as ``Servlet Filter’’ Approach Selector Parameters:  time_limit = 800ms,  reactivation_threshold = 400ms Approaches:  4 instances of a floating-point division servlet, configured to 100, 500, 1000 and 3000 loops.

11. (c) 2003 - Lindsay Bradford10 The Test Environment and Traffic Patterns: Response `adequate’ if <= 1 second round-trip recorded at client. –Steady – Responsiveness to constant load –Bursty – Responsiveness to variable load

12. (c) 2003 - Lindsay Bradford11 Bursty Pattern Results Unexpected high number of “heavy” approach attempts. Baseline is 3000 loop approach.

13. (c) 2003 - Lindsay Bradford12 Steady Pattern Results Again, Unexpected high number of “heavier” approach attempts.

14. (c) 2003 - Lindsay Bradford13 Conclusions: Benefit of Approach Selection outweighs its overhead. In both traffic patterns:  Returns more responses overall and significantly more within our one second target.  An unexpected high number of attempts at more costly approaches resulting in lower adequacy.

15. (c) 2003 - Lindsay Bradford14 Ongoing Work: Memory intensive servlet added  Similar results to CPU intensive servlet Varied Thread Numbers  Traffic pattern and approach matter. Varied Approach Selector Parameters  reactivation_threshold matters. time_limit no where near as much.

16. (c) 2003 - Lindsay Bradford15 Future Work: New I/O (Database simulation) servlet. Servlet Engine Modification.  Servlet specification is too limiting. Changing the Approach Selection Heuristic. Automated approach generation off baseline. Guidelines for automated service adaptation to request traffic.

17. (c) 2003 - Lindsay Bradford16 Finish. Questions? Suggestions?