Timecard: Controlling User-Perceived Delays in Server-Based Mobile Applications Lenin Ravindranath, Jitu Padhye, Ratul Mahajan, Hari Balakrishnan

Cloud Services Mobile Apps

Cloud Services Users are impatient – 100ms delay can cost substantial drop in revenue at Amazon – Similar observations from Google and Bing Response Time Matters

Control Variability in Response Times Request Response Fixed deadlines Trade-off quality for response time – More time to compute, better quality results – Flexible Services Deadline Server processing

The elephant is outside the room Request Response Server processing

The elephant is outside the room User click Server processing App Server Request Response Reading sensors Radio State (Radio wakeup) DNS and TCP connect Link quality (3G, HSPA+, LTE, Wifi) TCP state Datasize Phone model Parsing and Rendering UplinkDownlink Phone model User perceived delay Highly variable

No visibility into delays outside the service User click Server processing App Server Request Response Reading sensors Radio State (Radio wakeup) DNS and TCP connect Link quality (3G, HSPA+, LTE, Wifi) TCP state Datasize Phone model Parsing and Rendering UplinkDownlink Phone model User perceived delay Highly variable Unaware of end-to-end delays Clients with non-trivial external delays – Poor end-to-end response times Client with minimal external delays – Do not produce the best quality result Could adapt differently for different users

Timecard GetElapsedTime(); PredictDownstreamTime ( responseSize ); Time elapsed since user request Predicted downlink + app processing delay App Server

Timecard App Server Desired end-to-end delay App GetElapsedTime(); PredictDownstreamTime ( responseSize ); Adapt Processing Time

Timecard App Server Desired end-to-end delay App GetElapsedTime(); PredictDownstreamTime ( responseSize ); Adapt Response

Timecard App Server Desired end-to-end delay App GetElapsedTime(); PredictDownstreamTime ( responseSize );

Timecard GetElapsedTime(); PredictDownstreamDelay ( responseSize ); App Server

UI Thread Background Thread Server Threads UI dispatcher Thread start GPS start Web request GPS callback Event handler Spawn workers Request handler Send response Web callback App Server Challenges

UI Thread Background Thread App Server Challenges Transaction GetElapsedTime(); PredictDownstreamDelay ( responseSize ); No single reference clock Automatically collect data and learn

Online Transaction Tracking Time Synchronization Downstream Delay Predictor – Downlink delay – App processing delay Timecard

UI Thread Background Thread Server Threads App Server Transaction Tracking TC Transaction Context - Timestamps, transaction/device/network information TC GetTransaction(). GetStartTime();

Periodic time sync probes from app to server – Find drift and offset between clocks Use server clock as reference clock – Client maps local time to server time Synchronizing Time Efficient technique for probing – Aware of the radio state and traffic – Minimal extra delays – Energy efficient

Predicting Downstream Time App Server Downlink time App processing time PredictDownstreamTime ( responseSize );

Predicting Downlink Time Data size 1 KB to 40 KB of data RTT matters than throughput – Predict RTT TCP window state matters – Multiple RTTs – Estimate TCP Window & number of RTTs Complicated by middleboxes Middlebox Read TCP state Estimate TCP state

Predicting Downlink Time Data size Model downlink delay – Recent RTT – Bytes already transferred – Response size – Network provider – Client OS Middlebox Estimate TCP state Error in cellular network – median 17 ms – 90 th percentile 86ms

Predicting App Processing Time Parsing and rendering depends on data size Model processing time as a function of – Response data size – Phone model 4.6% (8ms) median error, 10% in 90 th percentile

Timecard Implementation App Server Transaction Tracking Transaction Tracking Transaction Tracking Transaction Tracking Logging Predictor Time Sync Data Collection Automatic Instrumentation WP Apps ASP.NET services APIs

Timecard can help control end-to-end delay Modified two services (and two apps) to use timecard Mobile Ads Service

0.1% runtime overhead Less than 1% memory overhead – Garbage collection in idle time Less than 1% network overhead Negligible battery overhead Timecard Overhead

Timecard GetElapsedTime(); PredictDownstreamDelay ( responseSize ); App Server Adapt Processing Time Adapt Response Desired end-to-end delay

Apply Timecard to Timecard SOSP Deadline GetElapsedTime(); Adapt quality and scope of the paper PredictDownstreamTime ( scope );