Framebuffer Compression Using Dynamic Color Palettes (DCP) Ayub A. Gubran Tor M. Aamodt

Motivation: framebuffer life cycle Application 1Application 2Application 3 OS Graphics Libraries / Graphic API calls CPU (Software Rendering) / GPU (Hardware Rendering) Render Framebuffer 1 Render Framebuffer 2 Render Framebuffer 3 System Compositor (e.g., Android’s SurfaceFlinger) Display framebuffer Display Controller Device screen refresh rate (i.e., 60 FPS) Refresh rates vary by application and user activity Recompose after every framebuffer update GPU or CPU Composition OS Level App Level HW Level Legends Framebuffer Compression Using Dynamic Color Palettes2

Motivation: The cost of accessing the framebuffer Every frame is consumed/processed multiple times in multiple stages. Energy cost: off-chip memory. Framebuffer Compression Using Dynamic Color Palettes3

Motivation: Mobile Applications Usage Framebuffer Compression Using Dynamic Color Palettes4 1- Flurry Analytics: It’s an App World. The Web Just Lives in It.It’s an App World. The Web Just Lives in It

Framebuffer Compression for UI Apps Most of the time is spent on UI apps with simple content (solid backgrounds, text, icons…). Simple content can be compressed well using color palettes (dictionaries). But how we can accurately predict the right color palette for each frame before we start rendering it. Framebuffer Compression Using Dynamic Color Palettes5

Observation: Inter-frame Temporal Coherence Pixels color change (new content) Per pixel change (moving content) New web search Scrolling Loading BBC news Loading Amazon Framebuffer Compression Using Dynamic Color Palettes6 Web browsing (Chrome) Time (frames) % of screen size Mostly the same content

Dynamic Color Palettes (DCP) Framebuffer Compression Using Dynamic Color Palettes7

Compression Steps Frame 0 Frame 1 Frame 2 FVC Time Construct FVC from current frame’s pixel values Construct the CCD dictionary using the FVC values To memory CCD FVC To memory FVCCCD To memory Framebuffer access compressed using the CCD Framebuffer Compression Using Dynamic Color Palettes8 FVC: Frequent Values Collector CCD: Common Colors Dictionary

Compression Pipeline Framebuffer Compression Using Dynamic Color Palettes9 9 GPU Pixel tiles 1 CCD (Compress current tile) 2b Write a compressed tile Write a non- compressed tile Can compress all pixels in the current tile? Yes No 3 FVC (constructs a CCD for the next frame) 2a Send Tile Buffer compressed & non-compressed tiles of block Write to memory 4 Associative search. Evict & replace on miss. Size Lookup table

DCP Variations Improving compression rates:  DCP: A fixed size color palette.  Variable DCP: Varying the color palette size every frame using the feedback from the previous frame.  Adaptive DCP: Encoding tiles based on the frequencies of their colors. Reducing the cost of DCP: Lower sampling rates: Only sending every nth pixel to the FVC to construct the palette. Using the same palette for multiple frames. Framebuffer Compression Using Dynamic Color Palettes10

Results Framebuffer Compression Using Dynamic Color Palettes 11 UI and 2D applications 3D games RED: Locally compressing similar colors in a tile 2 RAS: A more general prediction based compression 3 ADCP 4.13:1 VDCP 3.5:1 DCP 2.86:1 RAS 2.5:1 RED 2.62:1 RAS Cost: using 64 FVC and CCD FVC: 456 bytes CCD: 264 bytes

Limitations and Potential 3D games: Complex color patterns. Color palettes perform poorly. Multiple render targets: Need to construct a palette for every target but this is uncommon in UI apps. Compressing non-color render targets in 3D games: We see potential in compressing some non-color targets. Framebuffer Compression Using Dynamic Color Palettes12

Thank you Questions? Framebuffer Compression Using Dynamic Color Palettes13

References 1- Flurry Analytics: It’s an App World. The Web Just Lives in It.It’s an App World. The Web Just Lives in It 2- NVIDIA, NVIDIA Tegra X1 Whitepaper. URL: international.download.nvidia.com/pdf/tegra/Tegra-X1-whitepaper- v1.0.pdf 3- RASMUSSON, J., HASSELGREN, J., AND AKENINE-MOLLER, T Exact and error-bounded approximate color buffer compression and decompression. In SIGGRAPH/EUROGRAPHICS Conference On Graphics Hardware: Proceedings of the 22nd ACM SIGGRAPH/EUROGRAPHICS symposium on Graphics hardware, vol. 4, 41–48. Framebuffer Compression Using Dynamic Color Palettes14

Backup slides Framebuffer Compression Using Dynamic Color Palettes15

Workloads AB: Angry Birds. PH: Android Phone. CH: Chrome. FB: Facebook. HO: Android Home Screen. KD: Amazon Kindle. MS: Android Messaging. ST: Android Settings. TW: Twitter. TR: Temple Run 2. GS: Gun Ship 2. NS: Need for Speed. FN: Fruit Ninja. Framebuffer Compression Using Dynamic Color Palettes16

Lower pixel sampling at the FVC

Using the same palette for multiple frames

ADCP Example Framebuffer Compression Using Dynamic Color Palettes19 CCD C0C1C2C3C4C5C6C7C0C1C2C3C4C5C6C {00,01} {0,1} { φ } {000,101} {110,010} {110,111} {C 0,C y } {C x,C 2 } {C 0, C 2 } {C 0, C 1 } {C 0, C 0 } {C 0, C 5 } {C 6, C 2 } {C 6, C 7 } {C 0, C y } {C x, C 2 } Tile colorsEncodingCSB

FVC Fidelity vs size