1. Framebuffer Compression Using Dynamic Color Palettes (DCP) Ayub A. Gubran Tor M. Aamodt

2. Motivation: framebuffer life cycle Application 1Application 2Application 3 OS Graphics Libraries / Graphic API calls CPU (Software Rendering) / GPU (Hardware Rendering) Render Framebuffer 1 Render Framebuffer 2 Render Framebuffer 3 System Compositor (e.g., Android’s SurfaceFlinger) Display framebuffer Display Controller Device Screen @ screen refresh rate (i.e., 60 FPS) Refresh rates vary by application and user activity Recompose after every framebuffer update GPU or CPU Composition OS Level App Level HW Level Legends Framebuffer Compression Using Dynamic Color Palettes2

3. Motivation: The cost of accessing the framebuffer Every frame is consumed/processed multiple times in multiple stages. Energy cost: off-chip memory. Framebuffer Compression Using Dynamic Color Palettes3

4. Motivation: Mobile Applications Usage Framebuffer Compression Using Dynamic Color Palettes4 1- Flurry Analytics: It’s an App World. The Web Just Lives in It.It’s an App World. The Web Just Lives in It

5. Framebuffer Compression for UI Apps Most of the time is spent on UI apps with simple content (solid backgrounds, text, icons…). Simple content can be compressed well using color palettes (dictionaries). But how we can accurately predict the right color palette for each frame before we start rendering it. Framebuffer Compression Using Dynamic Color Palettes5

6. Observation: Inter-frame Temporal Coherence Pixels color change (new content) Per pixel change (moving content) New web search Scrolling Loading BBC news Loading Amazon Framebuffer Compression Using Dynamic Color Palettes6 Web browsing (Chrome) Time (frames) % of screen size Mostly the same content

7. Dynamic Color Palettes (DCP) Framebuffer Compression Using Dynamic Color Palettes7

8. Compression Steps Frame 0 Frame 1 Frame 2 FVC Time Construct FVC from current frame’s pixel values Construct the CCD dictionary using the FVC values To memory CCD FVC To memory FVCCCD To memory Framebuffer access compressed using the CCD Framebuffer Compression Using Dynamic Color Palettes8 FVC: Frequent Values Collector CCD: Common Colors Dictionary

9. Compression Pipeline Framebuffer Compression Using Dynamic Color Palettes9 9 GPU Pixel tiles 1 CCD (Compress current tile) 2b Write a compressed tile Write a non- compressed tile Can compress all pixels in the current tile? Yes No 3 FVC (constructs a CCD for the next frame) 2a Send Tile Buffer compressed & non-compressed tiles of block Write to memory 4 Associative search. Evict & replace on miss. Size 16-128 Lookup table

10. DCP Variations Improving compression rates:  DCP: A fixed size color palette.  Variable DCP: Varying the color palette size every frame using the feedback from the previous frame.  Adaptive DCP: Encoding tiles based on the frequencies of their colors. Reducing the cost of DCP: Lower sampling rates: Only sending every nth pixel to the FVC to construct the palette. Using the same palette for multiple frames. Framebuffer Compression Using Dynamic Color Palettes10

11. Results Framebuffer Compression Using Dynamic Color Palettes 11 UI and 2D applications 3D games RED: Locally compressing similar colors in a tile 2 RAS: A more general prediction based compression 3 ADCP 4.13:1 VDCP 3.5:1 DCP 2.86:1 RAS 2.5:1 RED 2.62:1 RAS Cost: using 64 FVC and CCD FVC: 456 bytes CCD: 264 bytes

12. Limitations and Potential 3D games: Complex color patterns. Color palettes perform poorly. Multiple render targets: Need to construct a palette for every target but this is uncommon in UI apps. Compressing non-color render targets in 3D games: We see potential in compressing some non-color targets. Framebuffer Compression Using Dynamic Color Palettes12

13. Thank you Questions? Framebuffer Compression Using Dynamic Color Palettes13

14. References 1- Flurry Analytics: It’s an App World. The Web Just Lives in It.It’s an App World. The Web Just Lives in It 2- NVIDIA, 2015. NVIDIA Tegra X1 Whitepaper. URL: international.download.nvidia.com/pdf/tegra/Tegra-X1-whitepaper- v1.0.pdf 3- RASMUSSON, J., HASSELGREN, J., AND AKENINE-MOLLER, T. 2007. Exact and error-bounded approximate color buffer compression and decompression. In SIGGRAPH/EUROGRAPHICS Conference On Graphics Hardware: Proceedings of the 22nd ACM SIGGRAPH/EUROGRAPHICS symposium on Graphics hardware, vol. 4, 41–48. Framebuffer Compression Using Dynamic Color Palettes14

15. Backup slides Framebuffer Compression Using Dynamic Color Palettes15

16. Workloads AB: Angry Birds. PH: Android Phone. CH: Chrome. FB: Facebook. HO: Android Home Screen. KD: Amazon Kindle. MS: Android Messaging. ST: Android Settings. TW: Twitter. TR: Temple Run 2. GS: Gun Ship 2. NS: Need for Speed. FN: Fruit Ninja. Framebuffer Compression Using Dynamic Color Palettes16

17. Lower pixel sampling at the FVC

18. Using the same palette for multiple frames

19. ADCP Example Framebuffer Compression Using Dynamic Color Palettes19 CCD C0C1C2C3C4C5C6C7C0C1C2C3C4C5C6C7 000 001 010 011 100 101 110 111 {00,01} {0,1} { φ } {000,101} {110,010} {110,111} {C 0,C y } {C x,C 2 } {C 0, C 2 } {C 0, C 1 } {C 0, C 0 } {C 0, C 5 } {C 6, C 2 } {C 6, C 7 } {C 0, C y } {C x, C 2 } 010 001 000 011 111 Tile colorsEncodingCSB

20. FVC Fidelity vs size