Lighting up HDR and advanced color in DirectX Simon Tao Program Manager Windows Graphics

What is advanced color? 1/27/2018 7:23 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

What is advanced color? Advanced color content provides… better dynamic range + better color gamut + better precision Advanced color displays can reproduce the above

Higher dynamic range a.k.a. “HDR”, “high color” Direct sunlight: >millions of nits Higher dynamic range a.k.a. “HDR”, “high color” More contrast between highlights and shadows SDR display: ~0.5 to 500 nits Metallic glint: 10,000 nits Shadows: less than 1 nit

(human visible colors) Spectral locus (human visible colors) Wider color gamut a.k.a. “WCG”, “wide color” More saturated hues sRGB: ~35% of human vision sRGB gamut

Deeper precision Reduces banding artifacts 2 bits per channel (simulated) 8 bits per channel (SDR display) Deeper precision a.k.a. “deep color”, “high bit depth” Reduces banding artifacts Human vision needs at least 10-12 bpc

What is advanced color? Advanced color content provides… Greater than: 300 nits peak lumi. sRGB color primaries 8 bits per channel Advanced color content provides… better dynamic range better color gamut better precision Advanced color displays can reproduce the above

Advanced color is a generational leap in visual fidelity

Light up your content and creativity Movies and videos Premium streaming services UHD Blu-Ray Video games Realistic, immersive visuals Physically-based rendering Creative apps Photo, video and content editing

How to use advanced color

System requirements Windows 10 Creators Update HDR TV or PC monitor HDR10 compliant High-end discrete GPU + WDDM 2.2 driver e.g. AMD Polaris, NVidia Pascal PlayReady 3.0 / HEVC 10b / HDCP 2.2 for premium video Advanced color automatically lights up on the display Turn on/off in Settings > System > Display Stay tuned!

Supported APIs in the Creators Update DirectX Direct3D11 Direct3D12 Direct2D Windows Runtime Win2D Windows.UI.Composition XAML <MediaElement> and <SwapChainPanel> HTML <video> This video covers these APIs

Deep dive: advanced color in DirectX

Terminology Pixel format: how your numbers are laid out in memory e.g. DXGI_FORMAT_R8G8B8A8_UNORM Colorspace: how you interpret the numbers as real colors e.g. sRGB, scRGB, HDR10 7 6 5 4 3 2 1 Red Green Blue Alpha RGB(0, 120, 215) = in the sRGB colorspace

Use an AC pixel format + colorspace Pixel format: more precision vs. UINT8 Colorspace: higher + wider colors vs. sRGB Two recommendations depending on scenario Option 1: FP16 + scRGB DXGI_FORMAT_R16G16B16A16_FLOAT DXGI_COLOR_SPACE_RGB_FULL_G10_NONE_P709 Option 2: R10G10B10A2 + HDR10 DXGI_FORMAT_R10G10B10A2_UNORM DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020

Use an AC pixel format + colorspace Option 1: FP16 + scRGB Good general purpose option Enough precision + range to represent “any” color Consumes 2x memory/bandwidth vs. RGBA8 Color behavior is different than sRGB Much of your shader and color math must be updated This applies to both Option 1 and 2 This is important!

Use an AC pixel format + colorspace Option 2: R10G10B10A2 + HDR10 Use this as an optimization for specific cases Recommended for windowed fullscreen scenarios Consumes same memory/bandwidth as RGBA8 GPU can eliminate some colorspace conversions Not recommended as an intermediate format Insufficient precision (especially alpha) Consider DXGI_FORMAT_R11G11B10_FLOAT for intermediates Not supported by Direct2D or Win2D

Use an AC pixel format + colorspace ComPtr<IDXGIFactory2> factory; // Initialized elsewhere DXGI_SWAP_CHAIN_DESC1 swapchainDesc = {}; swapchainDesc.Format = DXGI_FORMAT_R16G16B16A16_FLOAT; // Set pixel format ... ComPtr<IDXGISwapChain1> swapchain; ThrowIfFailed(factory->CreateSwapChainForCoreWindow( // Create swapchain &swapchainDesc, &swapchain)); ComPtr<IDXGISwapChain4> swapchain4; // QueryInterface swapchain.As(&swapchain4); swapchain4->SetColorSpace1(DXGI_COLOR_SPACE_RGB_FULL_G10_NONE_P709); // Set colorspace

Detect + adapt to display performance Display performance: color / luminance / precision Expect significant diversity across displays Expect realtime changes to display state Default to full quality and degrade gracefully for the display AC visual assets AC render pipeline AC display output SwapChain Alternate asset sets Power/perf optimizations Tonemapping Detect display capabilities

Detect + adapt to display performance ComPtr<IDXGIFactory1> factory; // Initialized elsewhere ComPtr<IDXGISwapChain> swapchain; // Initialized elsewhere if (!factory->IsCurrent()) { CreateDXGIFactory2(0 /* flags */, IID_PPV_ARGS(&factory)); // Ensure DXGI factory is valid } ComPtr<IDXGIOutput> output; swapchain->GetContainingOutput(&output); // Get DXGI output ComPtr<IDXGIOutput6> output6; output.As(&output6); // QueryInterface DXGI_OUTPUT_DESC1 outputDesc; output6->GetDesc1(&outputDesc); // Get display desc // DXGI_OUTPUT_DESC1 contains all key display color performance values

Detect + adapt to display performance typedef struct DXGI_OUTPUT_DESC1 { ... UINT BitsPerColor; // Precision DXGI_COLOR_SPACE_TYPE ColorSpace; // Display colorspace FLOAT RedPrimary[2]; // Color gamut FLOAT GreenPrimary[2]; FLOAT BluePrimary[2]; FLOAT WhitePoint[2]; FLOAT MinLuminance; // Dynamic range FLOAT MaxLuminance; FLOAT MaxFullFrameLuminance; };

Use advanced color image assets Direct3D HDR textures: use DDS BC6H User-visible HDR images: use JPEG XR Fully supports floating point, alpha, lossless SDR displays will render sRGB colors with clamping Other formats also work well OpenEXR (floating point HDR) JPEG w/ ICC profile (8 bit WCG only) MS is working with MPEG on future HDR image formats

Use advanced color image assets // Use Windows Imaging Component to load a JPEG XR image into Direct2D ComPtr<ID2D1DeviceContext2> context; // Initialized elsewhere ComPtr<IWICImagingFactory> factory; // Initialized elsewhere ComPtr<IStream> stream; // Initialized elsewhere ComPtr<IWICBitmapDecoder> decoder; ComPtr<IWICBitmapFrameDecode> frame; // Create WIC decoder factory->CreateDecoderFromStream(stream.Get(), ..., &decoder); // WIC detects codec from the stream decoder->GetFrame(0, &frame); // JPEG XR has 1 frame ComPtr<IWICFormatConverter> converter; factory->CreateFormatConverter(&converter); // Ensure FP16 pixel format converter->Initialize(frame.Get(), GUID_WICPixelFormat64bppPRGBAHalf, ...); ComPtr<ID2D1ImageSourceFromWic> d2dImage; // Create D2D image source context->CreateImageSourceFromWic(converter.Get(), &d2dImage);

Wrap up

Key takeaways Advanced color displays are the future Get an HDR monitor and GPU Build/update your DirectX apps with advanced color Adapt to the display’s color capabilities

Thank you! Resources and links Intro article about HDR: D3D12HDR SDK sample: ACPhotoViewer SDK sample: DirectX: Windows.UI.Composition: Win2D: DirectXTex: aka.ms/esmdsk aka.ms/nokc3a aka.ms/m39e01 aka.ms/ds3xd7 aka.ms/lxde6g github.com/Microsoft/Win2D github.com/Microsoft/DirectXTex

1/27/2018 7:23 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.