1. Feedback directed optimization in Compaq’s compilation tools for Alpha
Robert Cohn
P. Geoffrey Lowney
Compaq Computer Corporation

2. Feedback directed optimization
Compilers
profiles used to determine frequently executed paths
optimization makes common paths fast
other paths might be slower
16-Nov-18
FDO workshop

3. Feedback directed optimization
Mature and powerful classical optimizer
leverage existing optimizations
Feedback directed optimizations:
Augment cost model with profile information
Simple feedback directed restructuring
enables classical optimizations
FDO 1% of compiler
16-Nov-18
FDO workshop

4. Feedback directed optimization in the tool chain
Compiler
Linker
Bin Opt
FrontEnd
Optimizer
CodeGen
Source  IL
IL  IL
IL  obj
obj  bin
bin  bin
inliner
tracer
switch
commando
loe
real flow
register
allocation
scheduling
layout
alignment
16-Nov-18
FDO workshop

5. Profile information Basic block counts
pixie: instrumentation
DCPI: statistical sampling
Call edge counts computed from basic block counts
Flow edge counts estimated from basic block counts
16-Nov-18
FDO workshop

6. Procedure inliner
Static heuristics: estimates benefit of inlining a call site:
code size, register pressure, constant arguments, number of static callers
Frequency of execution:
lower or raise desirability
number of dynamic callers
16-Nov-18
FDO workshop

7. Tracer Transforms complicated control flow to superblocks
single entrance, multiple exit code sequence
Benefit from larger superblocks:
bigger scheduling unit
isolation of infrequently executed paths
16-Nov-18
FDO workshop

8. Tracer: superblock formation
Pick trace N1,…, Nn
Change trace to superblock
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A B C D E
16-Nov-18
FDO workshop

9. Tracer: superblock formation
Pick trace N1,…, Nn
Change trace to superblock
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A B C D E
16-Nov-18
FDO workshop

10. Tracer: superblock formation
Pick trace N1,…, Nn
Change trace to superblock
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A B C1 C D E
16-Nov-18
FDO workshop

11. Tracer: superblock formation
Pick trace N1,…, Nn
Change trace to superblock
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A B C1 C D E
16-Nov-18
FDO workshop

12. Tracer: superblock formation
Pick trace N1,…, Nn
Change trace to superblock
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A B C1 C D E1 E
16-Nov-18
FDO workshop

13. Tracer: superblock formation
Pick trace N1,…, Nn
Change trace to superblock
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A B C1 C D E1 E
16-Nov-18
FDO workshop

14. Tracer: loop peeling Pull 1 or 2 iterations out of loop
Implemented as superblock formation
p = p->n;
if (p == a) goto L1;
do {p = p-> n;
} while (p != a);
return p;
p = p->n;
if (p == a) goto L1;
do {
p = p->n;
} while (p != a);
L1: return p;
16-Nov-18
FDO workshop

15. Tracer: loop peeling Pick trace N1,…, Nn Change trace to superblock A
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A 1 B 1 1 C
16-Nov-18
FDO workshop

16. Tracer: loop peeling Pick trace N1,…, Nn Change trace to superblock A
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A 1 B 1 1 C
16-Nov-18
FDO workshop

17. Tracer: loop peeling Pick trace N1,…, Nn Change trace to superblock
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A 1 1 B1 B 1 C
16-Nov-18
FDO workshop

18. Tracer: loop peeling Pick trace N1,…, Nn Change trace to superblock
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A 1 1 B1 B 1 C
16-Nov-18
FDO workshop

19. Tracer: loop peeling Pick trace N1,…, Nn Change trace to superblock
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A 1 1 B1 B B2 C 1
16-Nov-18
FDO workshop

20. Tracer: loop peeling Pick trace N1,…, Nn Change trace to superblock
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A 1 B1 B 1 B2 C 1
16-Nov-18
FDO workshop

21. Tracer: loop peeling Pick trace N1,…, Nn Change trace to superblock
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A 1 B1 B 1 B2 C 1 C1
16-Nov-18
FDO workshop

22. Tracer: loop peeling Pick trace N1,…, Nn Change trace to superblock
visit nodes N2,…, Nn
if > 1 predecessor
copy node and outgoing edges
redirect incoming trace edge to copy A 1 B1 B 1 B2 C 1 C1
16-Nov-18
FDO workshop

23. Commando loop optimization
Restructure loop
frequent paths are in inner loop
infrequent paths moved to outer loop
Create opportunities for classical opt.
loop invariant removal
register allocation
Generalization of superblock loop optimization
16-Nov-18
FDO workshop

24. Commando loop optimization
Make two loop bottoms
Redirect infrequent back edges to one and the rest to other
Add loop preheader
Infrequent loop bottom targets preheader
Frequent loop bottom targets loop top 42 Q 40 4 R S V W T U
16-Nov-18
FDO workshop

25. Commando loop optimization
Make two loop bottoms
Redirect infrequent back edges to one and the rest to other
Add loop preheader
Infrequent loop bottom targets preheader
Frequent loop bottom targets loop top 42 Q 40 4 R S V W T U H C
16-Nov-18
FDO workshop

26. Commando loop optimization
Make two loop bottoms
Redirect infrequent back edges to one and the rest to other
Add loop preheader
Infrequent loop bottom targets preheader
Frequent loop bottom targets loop top Q R S V W T U 42 4 40 H C
16-Nov-18
FDO workshop

27. Commando loop optimization
Make two loop bottoms
Redirect infrequent back edges to one and the rest to other
Add loop preheader
Infrequent loop bottom targets preheader
Frequent loop bottom targets loop top P Q R S V W T U 42 4 40 H C
16-Nov-18
FDO workshop

28. Commando loop optimization
Make two loop bottoms
Redirect infrequent back edges to one and the rest to other
Add loop preheader
Infrequent loop bottom targets preheader
Frequent loop bottom targets loop top P 4 Q R S V W T U 42 4 40 H C
16-Nov-18
FDO workshop

29. Commando loop optimization
Make two loop bottoms
Redirect infrequent back edges to one and the rest to other
Add loop preheader
Infrequent loop bottom targets preheader
Frequent loop bottom targets loop top P 4 82 Q R S V W T U 42 4 40 H C
16-Nov-18
FDO workshop

30. Commando loop optimization
Make two loop bottoms
Redirect infrequent back edges to one and the rest to other
Add loop preheader
Infrequent loop bottom targets preheader
Frequent loop bottom targets loop top P 4
Inner loop 82 Q R S V W T U 42 4 40 H C
16-Nov-18
FDO workshop

31. Code layout Place code to improve: Pettis and Hansen
instruction cache utilization
memory working set
instruction prefetch
Pettis and Hansen
Basic block chaining
Routine ordering
Routine splitting
16-Nov-18
FDO workshop

32. Switch statement optimization
C switch statement
test for most frequent case first
switch (a) {
case 1: return 3;
case 2: return 4;
case 4: return 5; }
if (a == 4) return 5;
else
switch (a) {
case 1: return 3;
case 2: return 4; }
16-Nov-18
FDO workshop

33. Evaluation DS20: SPECInt95 Aggressive optimization for baseline
500MHZ 21264
SPECInt95
train: train workload
time: ref workload
Aggressive optimization for baseline
Median of 9 runs
16-Nov-18
FDO workshop

34. Speedup by optimization
16-Nov-18
FDO workshop

35. Speedup for inlining
16-Nov-18
FDO workshop

36. Code layout
16-Nov-18
FDO workshop

37. Tracer
16-Nov-18
FDO workshop

38. Commando
16-Nov-18
FDO workshop

39. Loop unroller
16-Nov-18
FDO workshop

40. Switch optimization
16-Nov-18
FDO workshop

41. Code growth by optimization
16-Nov-18
FDO workshop

42. Summary and conclusions
FDO is effective: 17% speedup
Complement to a strong classical optimizer
augment cost model of static optimization
simple restructuring transformations
Inlining is most important
Reduces code size
16-Nov-18
FDO workshop

43. Acknowledgements Gene Albert
Michael Adler, David Blickstein, Peter Craig, Caroline Davidson, Neil Faiman, Kent Glossop, David Goodwin, Rich Grove, Lucy Hamnett, Steve Hobbs, Bob Nix, Bill Noyce, and John Pieper
16-Nov-18
FDO workshop