Request Behavior Variations Kai Shen University of Rochester ASPLOS 2010 4/10/2019

Background Research Overview Complex request behaviors in server systems Potentially high concurrency in servers Fine-grained request activities over multiple stages Non-determinism due to multicore resource sharing Goal: Characterize/manage fine-grained request behaviors Model request performance (Cycles/ins.) and resource usage (to shared cache and memory bandwidth) Online signature identification and anomaly analysis Adaptive request scheduling and throttling Operating system techniques on existing hardware ASPLOS 2010 4/10/2019

A Request’s Life RUBiS: multi-stage online auction Web server J2EE application components Database A few milliseconds or less at each stage ASPLOS 2010 4/10/2019

Request Behavior Variations Performance obfuscation of multicore resource sharing Intel 3.0GHz “Woodcrest” processors One core Four cores ASPLOS 2010 4/10/2019

Intra-Request Behavior Variations Fluctuating behaviors over the course of one request execution Opportunties for signature identification and contention management ASPLOS 2010 4/10/2019

Related Work and Challenges Past work on phase identification/exploitation Program analysis: [Batson&Madison 76], [Shen et al. 04] Architecture: [Dhodapkar&Smith 02], [Sherwood et al. 03], [Isci&Martonosi 06] Challenges for characterizing request behavior variations High concurrency, frequent net/storage I/Os, and multiple-stage exec. in server applications  fine-grained patterns, no long stable phases OS-level techniques requiring no application changes or special hardware assistance Online characterization with low overhead, like in some past work ASPLOS 2010 4/10/2019

Our Approach Online request context maintenance [ASPLOS’08] Tracking request context propagation by tagging messages passed between server stages Online sampling of hardware event counters Sample event counters at request switches Sample at periodic interrupts to capture variation patterns As frequent as 10 microsecs per sample Observer’s effect An interrupt costs ~2300 cycles, triggers 12 references to L2 ASPLOS 2010 4/10/2019

System Call-Triggered Sampling In-kernel call much cheaper than interrupt Frequent system calls in servers present in-kernel opportunities ASPLOS 2010 4/10/2019

Behavior Transition Signals System calls are part of application semantics Some may signal impending behavior transitions Sample at these occasions more cost-effective Example: Apache web server Read file from disk  output HTTP headers to net  output file Output headers involve fragmented memory accesses  poor CPI signaled by a writev system call How to learn this? Collect sample system calls and metric changes before/after call On average, writev system call leads to CPI increase of 3.66 ASPLOS 2010 4/10/2019

Modeling: Request Differencing Quantify the similarity/difference of two requests Time shifting (due to non-deterministic multicore executions?) Dynamic time warping (originated in speech recognition) Time shifting ASPLOS 2010 4/10/2019

Modeling: Anomaly Detection Anomalies due to effects of dynamic resource contention Requests with similar L2 references/ins. but different Cycles/ins. Suspected contention ASPLOS 2010 4/10/2019

Utilization: Multicore Scheduling Contention-easing scheduling Avoid co-execution of high-resource-usage request periods Challenges Require prediction of fine-grained request behavior patterns Lack of long stable phases Much shorter than typical CPU scheduling quantum, in millisecs Results Able to reduce worst-case contention Reduce high-resource-usage co-execution at all four cores by 25% Little effect on average performance ASPLOS 2010 4/10/2019

Conclusions Request characterization helps request-level management Challenges: high concurrency, fine granularity, multicore obfuscation Our contributions Efficient online request behavior sampling Demonstrate utilizations in request modeling and management OS-level techniques requiring no application changes or special hardware assistance Future work: End-to-end request behavior analysis Correlate with program/application info to answer high-level questions Request characterization over distributed server systems ASPLOS 2010 4/10/2019