Guiding Ispike with Instrumentation and Hardware (PMU) Profiles CGO’04 Tutorial 3/21/04 CK. Luk Massachusetts Microprocessor Design Center Intel Corporation

CGO’04 Tutorial2 What is Ispike? A post-link optimizer for Itanium/Linux –No source code required –Memory-centric optimizations: Code layout + prefetching, data layout + prefetching –Significant speedups over compiler-optimized programs: 10% average speedup over gcc –O3 on SPEC CINT 2000 Profile usages: –Understanding program characteristics –Driving optimizations automatically –Evaluating the effectiveness of optimizations

CGO’04 Tutorial3 Profiles used by Ispike GranularityHardware Profiles (pfmon) Instrumentation Profiles (pin) Usages Per inst.PC sample ---Identifying hot spots Per inst. lineI-EAR (I-Cache) ---Inst. prefetching I-EAR (I-TLB) --- Per branchBTBEdge profileCode layout, data layout, and other opts Per loadD-EAR (D-Cache)Load-latency profileData prefetching D-EAR (D-TLB)--- D-EAR (stride)Stride profileData prefetching

CGO’04 Tutorial4 Profile Example: D-EAR (cache) Top 10 loads in the D-EAR profile of the MCF benchmark latency buckets Total sampled miss latency

CGO’04 Tutorial5 Profile Analysis Tools A set of tools written for visualizing and analyzing profiles, e.g.,: –Control flow graph (CFG) viewer –Code-layout viewer –Load-latency comparator

CGO’04 Tutorial6 CFG Viewer  For evaluating the accuracy of profiles

CGO’04 Tutorial7 Code-layout Viewer  For evaluating code-layout optimization

CGO’04 Tutorial8 Load-latency Comparator  For evaluating data-layout optimization and data prefetching

CGO’04 Tutorial9 Deriving New Profiles from PMUs New profile types can be derived from PMUs Two examples: –Consumer stall cycles –D-cache miss strides

CGO’04 Tutorial10 Consumer Stall Cycles Question: –How many cycles of stall experienced by I2 ? (Note: not necessarily the load latency of I1 ) Method: –PC-sample count is proportional to (stall cycles * frequency) I1: ld8 r2 = [r3];; /* other instructions */ I2: add r2 = r2, 1;; I3: st8 [r3] = r2 PC-sample count N1N1 N2N2 N3N3 Basic block A

CGO’04 Tutorial11 D-cache Miss Strides Problem: –Detect strides that are statically unknown arc* arcin; node* tail; … while (arcin) { tail = arcin->tail; … arcin = tail->mark; } arcin tail -192B -120B Example: Two strided loads in MCF

CGO’04 Tutorial12 D-EAR based Stride Profiling Sample load misses with 2 phases: Time Skipping phases (1 sample per 1000 misses) Inspection phases (1 sample per miss) GCD(A 2 -A 1, A 3 -A 2 )=GCD(240,336)=48 GCD(A 3 -A 2, A 4 -A 3 )=GCD(336,144)=48 Use GCD to figure out strides from miss addresses: Time A1A1 A2A2 A3A3 A 2 -A 1 =5*48=240A 3 -A 2 =7*48=336 A4A4 A 4 -A 3 =3*48=144 A 1, A 2, A 3, A 4 are four consecutive miss addresses of a load. The load has a stride of 48 bytes.

CGO’04 Tutorial13 Performance Evaluation Instrumentation vs. PMU profiles: –Profiling overhead –Performance impact Ispike optimizations: –Code layout, instruction prefetching, data layout, data prefetching, inlining, global-data optimization, scalar optimizations Baseline compilers: –Intel Electron compiler (ecc), version 8.0 Beta, -O3 –GNU C compiler (gcc), version 3.2, -O3 Benchmarks: –SPEC CINT2000 (profiled with “training”, measured with “reference”) System: –1GHz Itanium 2, 16KB L1I/16KB L1D, 256KB L2, 3MB L3, 16GB memory –Red Hat Enterprise Linux AS with kernel

CGO’04 Tutorial14 Performance Gains with PMU Profiles Up to 40% gain Geo. means: 8.5% over Ecc and 9.9% over Gcc Ecc8.0 –O3 baseline Gcc3.2 –O3 baseline BTB (1 sample/10K branches), D-EAR cache (1 sample/100 load misses) D-EAR stride (1 sample /100 misses in skipping, 1 sample/miss in inspection)

CGO’04 Tutorial15 Cycle Breakdown (Ecc Baseline)  Help understand if individual optimizations are doing a good job

CGO’04 Tutorial16 PMU Profiling Overhead Overhead reduced from 58% to 23% when lowering the BTB sampling rate by 10x. Overhead reduced to 3% when lowering the D-EAR sampling rate by 10x.

CGO’04 Tutorial17 Instrumentation Profiling Overhead Why is the overhead so large? –Training runs are too short to amortize the dynamic compilation cost –Techniques like ephemeral instrumentation yet to be applied

CGO’04 Tutorial18 PMU vs. Instrumentation (Perf. Gains) PMU profiles can be as good as instrumentation profiles –Could be even better in some cases (e.g., mcf ) However, possible performance drops when samples are too sparse –E.g., gap and parser when Stride = profiling overhead >60x 59% 24% 3%

CGO’04 Tutorial19 Reference “Ispike: A Post-link Optimizer for the Intel Itanium Architecture”, by Luk et. al. In Proceedings of CGO’04.