Scaling of Eclipse on High Density Displays Sravan Kumar Lakkimsetti Committer SWT Project IBM India

Agenda What are High Density (HiDPI) displays? How is Eclipse impacted by HiDPI? Requirements Native OS level support Current and Proposed Solutions Solution from Eclipse/SWT Different Approaches Implementation details from SWT New APIs introduced and usage 9/17/2018

What are High Density displays Pixel density is measured by number of pixels/dots in an inch (PPI/DPI). Increasing with each year to get more clarity on the displays. 100% scaling levels for different operating systems Mac OS X - 72 dpi Windows - 96 dpi Linux/GTK+ - 96 dpi SWT considers displays with a dpi more than 144 (150% for Win/GTK+ and 200% for Mac OS X ) as High Density displays. 9/17/2018

Problems at high DPI levels Increasing the dpi makes the UI elements appear smaller. Even though they are of same pixel size. Eclipse/SWT are expected to render UI elements in the same size independent of pixel grid. This way the UI is consistent across displays. 9/17/2018

Requirements Requirements Work with current OS SDK base line API for Windows, GTK+ and Mac OS X API baselines used Mac OS X – API v10.9 Windows – Windows SDK v7.0 GTK+– v3.0 Support Scaling factors of 100%, 150% and 200%. Need to have different resolution images stored in separate bundles. Load only the required images corresponding to the scaling factor Make all the graphics functionality dpi aware The graphics api should accept coordinates in points instead of pixels (1 point width at 200% scaling is equal to 2 pixels wide) Scale-up all the images that do not have corresponding high dpi images. 9/17/2018

OS level support (Mac OS X ) Mac OS X scales up all the UI elements to the user comfort level (by default enabled) Uses multiple representations to the same image at different dpi levels The measurement on display is in points (logical pixels). At 100% 1 point width is equal to 1 pixel wide and at 200% it will be 2 pixels Pros No need to make any changes in the code to support high DPI Cons The graphical UI elements appear blurred. Possibility of maintaining high number of bitmaps in the eclipse session 9/17/2018

OS level support (Windows) Scaling up all the UI elements can be done using Display properties Control Panel->Appearance and Personalization->Display Can set the scaling factor in 100%, 125%, 150% etc Only Dpi Aware applications can be scaled. Text scales up as it is supplied by the Windows font service Cons the text will scale up but others like icons, drawn lines etc would not scale 9/17/2018

OS level support (Windows) - contd At 100% scale level At 200% scale level 9/17/2018

OS level support (Ubuntu) User can specify the scale factor using Display properties Can set the scaling factor in multiples of 1/8. Text scales up as it is supplied by the font service Cons the text will scale up but others like icons etc would not scale (very similar to Windows) 9/17/2018

OS support On Ubuntu 100% scaling 200% Scaling 9/17/2018

Current and Proposed Solutions Mac OS X Have multiple bitmaps associated with the image object. OS will select the appropriate bitmap while rendering Works with integer multiples of scaling factor GTK3 Scale the Images and widgets Available from GTK 3.10 and above Works with integer multiples of the scale factor Java 9 JEP 251: Multi-Resolution Images JEP 263: HiDPI Graphics on Windows and Linux These approaches are similar to the Mac OS X approach. But due to multiple representations used, the memory foot print will be difficult to manage. 9/17/2018

SWT Approach 1 Use SVG format for the images and icons Add a new API addRepresentation to the Image class Use appropriate representation during the rendering of the Image Problems SVG images result in increase of eclipse SDK download by more than 100 MB Windows code does not handle SVG Each representation is a image handle at the OS level Resulted in high memory usage 9/17/2018

SWT Approach 2 Create the images when required at the corresponding dpi level Make the graphics API dpi aware. Use pixel geometry Problems Too many hacks had to be performed to use pixel geometry The behavior will not be consistent across all platforms 9/17/2018

SWT Approach 3 Change to point geometry The display geometry has been changed from pixel geometry to point geometry (similar to Mac OS X) All the coordinates in graphics API changed to point geometry All the events, sizes and locations now use point geometry Internally, SWT still uses pixel geometry to deal with OS level calls 9/17/2018

Implementation Detect the scaling factor and use while calculating the sizes in pixels Windows Use the GetDeviceCaps native API to get the dpi Linux Ubuntu Extract scaling factor from display settings Other Linux/UNIX platforms Calculate the dpi based on monitor dimensions 9/17/2018

Implementation (contd) Image Class Now the Image constructors accept sizes in points and create Image at appropriate sizes Introduced two new interfaces to have call backs while rendering ImageDataProvider getImageData(int zoom) ImageFileNameProvider getImagePath(int zoom) Image class has two new constructors Image(device, ImageDataProvider) Image(device, ImageFileNameProvider) Use these new constructors to have appropriate call backs at the time of rendering 9/17/2018

Implementation (contd) Widgets, Layouts and Resource classes Changed all the methods related to size and location calculations to use points instead of pixels All the existing public methods have been moved to *InPixels() methods New wrapper methods have been created to handle conversion from pixels to points 9/17/2018

Implementation (contd) GC class GC.drawImage() refreshes the bitmap before rendering Uses the ImageProvider API to make a call back to the client to get the appropriate bitmap and render it If not available the existing bitmap will be scaled appropriately and rendered Other public methods including draw methods Moved to *InPixels methods Added new wrapper methods to match existing public methods These wrappers provide the necessary conversion from pixels to points or points to pixels All these changes resulted in getting eclipse to render with a bigger UI on high DPI monitors! 9/17/2018

Lets see how Eclipse looks like on Ubuntu with all these changes!! 9/17/2018

New APIs ImageProviderAPI There are two ImageProvider interfaces provided for supplying the Images at higher level scaling ImageDataProvider getImageData(int zoom) ImageFileNameProvider getImagePath(int zoom) 9/17/2018

New APIs in Image Image.getBoundsInPixels() – returns image size in pixels Can be used while creating the ImageData, when used by the ImageDataProvider Image.getImageDataAtCurrentZoom() – returns image data that is used at current zoom level Useful when used with ImageDataProvider 9/17/2018

Test build is available at Next Steps Platform UI needs to be updated to use these new SWT APIs so that we can get sharper images Find a way to identify user provided scaling factor on gnome and use it Test build is available at https://drive.google.com/file/d/0B-IHCYXe3TALenBtcDl4QXVZS2M/view?usp=sharing 9/17/2018

Questions ???

9/17/2018

Evaluate the Sessions Sign in and vote at eclipsecon.org - 1 + 1