Offline Adaptation Using Automatically Generated Heuristics Frédéric de Mesmay, Yevgen Voronenko, and Markus Püschel Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, USA Marius Fehr, December 7 th

Online vs. Offline Adaptive Libraries 2 Online Adaptive Offline Adaptive Machine Learning d = dft(n) Search generated at Installation d(X,Y) – possibly unbounded initialization time – impractical for constantly changing problem specifications + bounded initialization time + no search if input specifications change

3 Motivation Background Adaptation Process Results

Statistical Classifier – C4.5 4 Source: Offline Adaptation Using Automatically Generated Heuristics, CMU features ˃Based on entropy, the measure of uncertainty ˃Feature with smallest entropy becomes root

DFT – Discrete Fourier Transforms ˃The Fourier Transform is a Linear Transform: 5 X X Y Y DFT n = *

FFT – Fast Fourier Transforms ˃Divide and Conquer Algorithms 6 Source: Lecture Slides :“How to Write Fast Numerical Code “ (ETH, CS), Markus Püschel

FFTW – Search space 7 dft ( 128 ) These decisions can be optimized! Radix 8 dft_strided (16, 8) dft_scaled (8, 16) Radix 4 No ˃Features :Problem size and stride ˃Decisions:Use of base cases and choice of radix Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

FFTW – Advanced Implementations 8 Lots of decisions! Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

9 Motivation Background Adaptation Process Results

Offline Adaptation Process 10 C4.5 Search x x x x x x x x x x x x x x Heuristic featuresdecisions sizestrideuse base caseradix 128-no8 16-no4 4-yes- 168no4 ………… Trainings Set

Offline Adaptation Process 11 Exploration Statistical Classification Verification Combination Online Adaptive Library Offline Adaptive Library Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

Exploration ˃“Creates a table for the statistical classifier to work with” 12 Exploration Statistical Classification Verification Combination Search x x x x x x x x x x x x x x featuresdecisions SizeStrideUse Base caseRadix 128-No8 16-No4 4-Yes- 168No4 ………… Trainings Set

Statistical Classification ˃Computes decision trees ˃Uses a modified version of C4.5 ˃Provides “hints” based on library functionality 13 Exploration Statistical Classification Verification Combination Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

Hinting ˃Problem:Sometimes C4.5 has not enough information to make wise decisions ˃Choice of radix: ˃Performance depends strongly on the prime factorization ˃Multiple of 2 and 3 show very different behavior but are heavily interleaved ˃Solution:Provide additional features to C Exploration Statistical Classification Verification Combination n n < … … … 58 > 59 Radix ?

Hinting 15 Exploration Statistical Classification Verification Combination Radix 12 Radix 3 Radix 18 Radix 6 Radix 4 nfactor( 2, n ) nfactor( 3, n ) n

16 Exploration Statistical Classification Verification Combination Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

Verification 17 Exploration Statistical Classification Verification Combination 24 = 2 * 2 * 2* 3 Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

Combination ˃Inserts decision trees into the library as heuristics 18 Exploration Statistical Classification Verification Combination Online Adaptive d = dft(n) Search d(X,Y) Offline Adaptive

19 Motivation Background Adaptation Process Results

2-Powers – Competitiveness 20 Platform: 2 x dual core 3GHz Intel Xeon 5160 Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

Learning and Generating Heuristics 21 Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

Mixed Sizes - Competitiveness 22 Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

˃Pro ˃Improvement of usability ˃Only small performance penalty (for DFT) ˃Entirely automatic method ˃Applicable to other problem domains / libraries ˃Computer generation of offline adaptive library, directly from algorithm specification (together with Spiral) ˃Contra ˃Performance depends strongly on choice of training set Pro vs. Contra Algorithm Specification SPIRAL this paper Adaptive Offline Library 23

Questions ? 24

Questions ? 25

Online vs. Offline Adaptive Libraries 3 Library TypeNon – adaptiveOnline adaptiveOffline adaptive PrototypeIPPFFTWthis paper Interface Initialization cost Computation cost Adaptation mechanism -online (planer at runtime) offline (at installation time) problem changes-rerun planer- platform changesrebuyrerun planerreinstall 26 Source: Offline Adaptation Using Automatically Generated Heuristics, CMU

Math behind C4.5 27