Mantis: Automatic Performance Prediction for Smartphone Applications Yongin Kwon, Sangmin Lee, Hayoon Yi, Donghyun Kwon, Seungjun Yang, Byung-Gon Chun, Ling Huang, Petros Maniatis, Mayur Naik, Yunheung Paek USENIX ATC’13

Performance Prediction Problem Predict the execution time of a program on a given input before running it.

Two kinds of Approaches Most existing techniques can be classified into two broad categories. ◦ Domain-specific programs, automatically- extracted features. ◦ General-purpose programs, manually-specified features.

Mantis A new framework to automatically predict the performance of general- purpose byte-code programs on given inputs. Four components: ◦ Feature instrumentor ◦ Profiler ◦ Performance model generator ◦ Predictor code generator

Architecture

Feature Instrumentor Instruments the program to collect the values of feature (f 1, …, f M ) as per feature schemes. Feature scheme ◦ Branch counts ◦ Loop counts ◦ Method-call counts ◦ Variable values

Examples

Profiler Outputs a data set ◦ t i : the i th observation of execution time. ◦ v i : the i th observation of the vector of M features.

Performance Modeling Performs a sparse nonlinear regression on the feature values and execution time. Produces a function ◦ is the approximation of execution time ◦ is a subset of  In practice, K << M.

Performance Modeling(Cont.) However, regression with best subset selection is NP-hard. ◦ Find the subset of size K that gives the smallest Residual Sum of Squares(RSS). ◦ Discrete optimization problem.

SPORE-FoBa Sparse POlynomial REgression – FoBa. ◦ A feature from the candidate set is added into the model if and only if adding it makes the RSS decrease a lot.  If the drop is greater than ε. ◦ Remove a feature from the active set if deleting it makes the RSS increase the least.  If the increment is smaller than ε ’.

Example Degree-2 polynomial with ◦ Expand (1+x 1 +x 2 ) 2 to get 1, x 1, x 2, x 1 2, x 1 x 2, x 2 2. ◦ Construct the following function for regression

Predictor code generator Produce a code snippet, called slice, for each chosen features. ◦ Slice: an executable sub-programs that yields the same value v of a feature at a program point p as the given program on all inputs. Automatically evaluate feature values for each input by executing slices.

Example

Prototype Toolchain

Experiment Setup A machine runs Ubuntu bit with a 3.1GHz quad-core CPU, and 8GB of RAM. A Galaxy Nexus running Android with dual-core 1.2Ghz CPU and 1GB RAM. Six CPU-intensive Android applications. ◦ Each with 1,000 randomly generated inputs. ◦ Train the predictor on 100 inputs.

Experimental Results

Features and Models

Effect of the Number of Training Inputs

Compare with Linear Model

Prediction Time of Mantis and PE

Prediction Error of Mantis and BE

Prediction on Different Hardware Platform

Prediction under Background Load

Offline Stage Processing Time

Conclusion Mantis is a framework that automatically generates program performance predictors. ◦ Combines program slicing and sparse regression in a novel way. Evaluation shows that the generated predictors estimate execution time accurately and efficiently for smartphone applications.