1. High Performance Linear Transform Program Generation for the Cell BE
Vas Chellappa
Franz Franchetti
Markus Püschel
Electrical & Computer Engineering
Carnegie Mellon University
Sponsors: DARPA-DESA, NSF, ARO, and Mercury Inc.

2. How do we harness the Cell’s impressive peak performance?
Cell Broadband Engine
Multicore cpu (8 SPEs+1 PPE)
SPEs: SIMD cores designed for numerical computing
256KB “local store” per SPE (scratchpad-like)
Programmer-driven DMA
204 Gflop/s peak
Cell BE Chip
Main Mem
EIB
SPE LS
How do we harness the Cell’s impressive peak performance?

3. DFT on the Cell BE
Spiral generated
(this paper)
350x
FFTC
FFTW
Numerical Recipes
Platform-tuned code is 350x faster. But hard to write!

4. Overview Background, Spiral Overview Generating DFTs for the Cell
Performance Results
Concluding Remarks
Markus Püschel, José M. F. Moura, Jeremy Johnson, David Padua, Manuela Veloso, Bryan Singer, Jianxin Xiong, Franz Franchetti, Aca Gacic, Yevgen Voronenko, Kang Chen, Robert W. Johnson, and Nick Rizzolo: SPIRAL: Code Generation for DSP Transforms. Special issue, Proceedings of the IEEE 93(2), 2005

5. “Fitting” Dataflow to Hardware
Core 0
Core 1
Parallel execution (multicore)
Stage 1
Stage 2
Stage 3
Stage 4
Iterative Algorithm (programming ease)
Stage 5
Stage 1
Recursive algorithm (memory hierarchy)
Stage 2
Stage 3
Stage 4
To “fit” DFT to architecture:
Various traversals
Various factorizations
How to map dataflow to architecture automatically?

6. “Fitting” Dataflow to Platform (contd.)1 2 3 4 5 1 2 3 4
Core 0
Core 1
Intuition: rewrite formulas to obtain suitable dataflow

7. Program Generation in Spiral
parallelization vectorization
loop optimizations
constant folding
scheduling ……
Optimization at all abstraction levels
Transform user specified
Fast algorithm in SPL many choices
∑-SPL
Iteration of this process
to search for the fastest
But that’s not all …
C Code

8. Common Abstraction: SPL
SPL: Tensor-product representation
Eg.: Cooley-Tukey fast Fourier transform (FFT):
Algorithms in SPL: Products of structured sparse matrices
Algorithms reduce arithmetic cost O(n2)  O(n log n)
Mathematical notation exposes structure: SPL (signal processing language)
Tensor products in SPL represent loop structures

9. Overview Background, Spiral Overview Generating DFTs for the Cell
Performance Results
Concluding Remarks

10. Mapping DFTs to the Cell
Objective: High-performance transform library for Cell BE
Cell BE Chip
Main Mem
EIB
SPE LS
DFT
Cell’s architectural paradigms:
Vectorize DFT for vector length 
Vectorization
Parallelize DFT across p SPEs, and use a DMA packet size of 
Parallelization
Optimize DFT for throughput (s DFTs required)
Multibuffering
Tags guide formula rewriting

11. Idea: rewrite all SPL constructs to parallel constructs + on-chip DMA
SPL to Parallel Code
Natural parallel construct in SPL: A x y
Processor 0
Processor 1
Processor 2
Processor 3
Independent, load-balanced, communication-free operation
Parallelizing other constructs in SPL:
Permutations require message exchange (on-chip DMA comm.) x y
Idea: rewrite all SPL constructs to parallel constructs + on-chip DMA

12. Idea: rewrite algorithm at SPL level to achieve largest DMA packets
SPL to Streaming Code
Streaming: Overlapping computation with communication
On-chip (SPE ↔ SPE) and off-chip (SPE ↔ Main memory)
Idea: tensor loops become multi-buffered loops
Useful for:
Throughput-optimized code
Large, out-of-chip sizes
i'th iteration
Write Ai-1
Compute Ai
Read Ai+1 A A A
(Trickier for other SPL constructs) x y
Idea: rewrite algorithm at SPL level to achieve largest DMA packets

13. Generating Cell Code Transform user specified Rewriting
Fast algorithm in SPL tag guided
Streamed from memory for throughput
Load balanced across p SPEs
SIMD kernel optimized for memory hierarchy
All-to-all communication (on-chip)
Loop operations in ∑-SPL
Cell-specific optimized C code (intrinsics, DMA etc.)

14. Generated Code Sample DFT 216: 4,000+ lines of code! vectorized DMA
/* Complex-to-complex DFT size 64 on 2 SPEs */ dft_c2c_64(float *X, float *Y, int spuid) { // Block 1 (IxA)L for(i:=0; i<=7; i++) // Right most gather { DMA_GATHER(gath_func(X,i), gath_func(T1,i), 4) } // uses spu_mfcdma() spu_mfcstat(MFC_TAG_UPDATE_ALL); // Wait on gather // compute vectorized DFT kernel of size m for(i:=0; i<=7; i++) // Scatter at interface { DMA_SCATTER(scat_func(T1,i), scat_func(T2,i), 4) } all_to_all_synchronization_barrier(); // uses mailbox msgs // Block 2 (AxI) /* Gather is a no operation since the scatter above accounted for it */ // compute vectorized DFT kernel of size n for(i:=0; i<=7; i++) // Left most scatter { DMA_SCATTER(scat_func(T1,i), scat_func(Y,i), 4) } all_to_all_synchronization_barrier(); }
vectorized
DMA
parallelized
DFT 216: 4,000+ lines of code!

15. Problem Space: Options
Parallelization
Base (Vectorized)
SPE
DFT
SPE
DFT
Vectorization assumed
Single DFT parallelized across multiple SPEs
SPE
DFT
Main Memory Operations
(Only for small DFTs)
SPE
DFT
Multiple independent DFTs on multiple SPEs
Latency optimized
(default)
SPE
DFT
SPE
DFT
Multiple parallelized independent DFTs
Throughput,
multibuffered

16. Problem Space: Combinations
Throughput-optimized usage scenarios
Latency-optimized usage scenarios
SPE
DFT
Parallel, multibuffered DFT
Single DFT from main memory
Independent DFTs multibuffered in parallel
Devise rewrite rules for tags. Nestings describe all scenarios

17. Overview Background, Spiral Overview Generating DFTs for the Cell
Performance Results
Concluding Remarks

18. SPE
DFT
8-SPEs
4-SPEs
2-SPEs
Single precision
IBM QS22
1-SPE

19. 4.5x faster than FFTW, 1.63x faster than FFTC
SPE
DFT
Spiral: 1-SPE
Spiral: 8-SPEs
FFTC
FFTW
4.5x faster than FFTW, 1.63x faster than FFTC

20. More Performance Results
Single-SPE DFT code
Split/interleaved complex formats
Non-2-power sizes
Double precision (PowerXCell 8i)
Mercury
Spiral
Chow
IBM SDK

21. Other Linear Transforms
Discrete Sine, Cosine transforms, DFT with real inputs (single-SPE)
2-D DFTs
Out-of-core sizes
Limited to 2D DFTs on 1-SPE (for now)
More performance results:
Srinivas Chellappa, Franz Franchetti , and Markus Püschel: Computer Generation of fast Fourier Transforms for the Cell Broadband Engine Proceedings of International Conference on Supercomputing (ICS) 2009

22. Overview Background, Spiral Overview Generating DFTs for the Cell
Performance Results
Concluding Remarks

23. Conclusion Automatic generation of transform libraries
High performance
Variety of scenarios, formats
High performance on Cell requires:
Vectorization multi-core parallelization, streaming, DMA code
Future processors likely to have similar paradigms, tradeoffs
Spiral approach:
Common abstraction of transform, algorithm, architecture (SPL)
Rewrite rules to go from transform to architecture
architecture
space
algorithm