Timecard: Controlling User-Perceived Delays in Server-Based Mobile Applications Lenin Ravindranath, Jitu Padhye, Ratul Mahajan, Hari Balakrishnan

Cloud Services Mobile Apps

Cloud Services Users are impatient – 100ms delay can cost substantial drop in revenue at Amazon – Similar observations from Google and Bing Response Time Matters

Control Variability in Response Times Request Response Fixed deadlines Trade-off quality for response time – More time to compute, better quality results – Flexible Services For e.g., search Deadline Server processing

The elephant is outside the room Request Response Server processing

The elephant is outside the room User click Server processing App Server Request Response Reading sensors Link quality (3G, HSPA+, LTE, Wifi) Response size Phone model Parsing and Rendering UplinkDownlink User perceived delay Highly variable Radio State

The elephant is outside the room User click Server processing App Server Request Response Reading sensors Link quality (3G, HSPA+, LTE, Wifi) Response size Phone model Parsing and Rendering UplinkDownlink User perceived delay Highly variable Radio State Unaware of end-to-end delays Clients with non-trivial external delays – Poor end-to-end response times Client with low external delays – Do not produce the best quality result

The elephant is outside the room User click Server processing App Server Request Response Reading sensors Link quality (3G, HSPA+, LTE, Wifi) Response size Phone model Parsing and Rendering UplinkDownlink User perceived delay Highly variable Radio State Servers should adapt to external delays and control end-to-end delay variability

Timecard GetElapsedTime(); PredictRemainingTime ( responseSize ); Time elapsed since user request Predicted downlink + app processing delay App Server

Timecard App Server Desired end-to-end delay App GetElapsedTime(); PredictRemainingTime ( responseSize ); Adapt Processing Time

Timecard App Server Desired end-to-end delay App GetElapsedTime(); PredictRemainingTime ( responseSize ); Adapt Response

Timecard App Server Desired end-to-end delay App GetElapsedTime(); PredictRemainingTime ( responseSize );

Timecard GetElapsedTime(); PredictRemainingTime ( responseSize ); App Server

UI Thread Background Thread Server Threads UI dispatcher Thread start GPS start Web request GPS callback Event handler Spawn workers Request handler Send response Web callback App Server Challenges

UI Thread Background Thread App Server Challenges Transaction GetElapsedTime(); PredictRemainingTime ( responseSize ); No single reference clock Automatically collect data and learn

Online Transaction Tracking Time Synchronization Remaining Time Predictor – Downlink delay – App processing delay Timecard

Periodic time sync probes from app to server – Find drift and offset between clocks Use server clock as reference clock – Client maps local time to server time Synchronizing Time Efficient technique for probing – Aware of the radio state and traffic – Minimal extra delays – Energy efficient

Predicting Remaining Time App Server Downlink time App processing time PredictRemainingTime ( responseSize );

Predicting Downlink Time Data size 1 KB to 40 KB of data RTT matters than throughput – Predict RTT TCP window state matters – Multiple RTTs – Estimate TCP Window & number of RTTs Complicated by middleboxes Middlebox Read TCP state Estimate TCP state

Predicting Downlink Time Data size Model downlink delay – Recent RTT – Bytes already transferred – Response size – Network provider – Client OS Middlebox Estimate TCP state Error in cellular network – median 17 ms – 90 th percentile 86ms

Predicting App Processing Time Parsing and rendering depends on data size Model processing time as a function of – Response data size – Phone model 4.6% (8ms) median error, 10% in 90 th percentile

Timecard Implementation App Server Transaction Tracking Transaction Tracking Transaction Tracking Transaction Tracking Logging Predictor Time Sync Data Collection Automatic Instrumentation WP Apps ASP.NET services APIs

Timecard can help control end-to-end delay Modified two services (and two apps) to use timecard Mobile Ads Service

0.1% runtime overhead Less than 1% memory overhead – Garbage collection in idle time Less than 1% network overhead Negligible battery overhead Timecard Overhead

Timecard GetElapsedTime(); PredictRemainingTime ( responseSize ); App Server Desired end-to-end delay Adapt Processing Time Adapt Response

Timecard GetElapsedTime(); PredictRemainingTime ( responseSize ); App Server Request Prioritization Desired end-to-end delay

Timecard GetElapsedTime(); PredictRemainingTime ( responseSize ); App Server Adapt Resources Used Desired end-to-end delay

Backup

UI Thread Background Thread Server Threads App Server Transaction Tracking TC Transaction Context - Timestamps, transaction/device/network information TC GetTransaction(). GetStartTime();