Swing 2 JavaFX Steve Northover (Oracle) Anton Taraov (Oracle)

The following is intended to outline our general product direction The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Program Agenda New and Exciting: SwingNode and Unified Threading! Old and not so Exciting: Where we are Today Discussion Topics: What should we do next?

SwingNode JFXPanel (JavaFX 2.0) SwingNode (JavaFX 8.0) augments existing Swing based apps with FX features the basis of an incremental migration strategy SwingNode (JavaFX 8.0) a reverse direction bridge to Swing takes in legacy/commercial Swing components to modern FX apps

API As simple as: package javafx.embed.swing; public class SwingNode extends Node { public void setContent(final JComponent content); public JComponent getContent(); } Thread safe JComponent : no hw components in the hierarchy, please

Threading Restrictions Invoke a UI related method on the right event thread: Swing on Event Dispatch thread: SwingUtilities.invokeLater(Runnable r); FX on JavaFX Application thread: Platform.runLater(Runnable r); SwingNode belongs to FX! Exclusion: SwingNode.setContent/getContent are thread safe methods

Code Sample public class SwingInFx extends Application { @Override public void start(Stage stage) { final SwingNode swingNode = new SwingNode(); createSwingContent(swingNode); Group group = new Group(); group.getChildren().add(swingNode); stage.setScene(new Scene(group, 100, 50)); stage.show(); } private void createSwingContent(final SwingNode swingNode) { SwingUtilities.invokeLater(() -> swingNode.setContent(new JButton("Click me!")));

Approach Symmetrical to JFXPanel: SwingNode is a first class FX node: lightweight embedding events are forwarded from FX to Swing SwingNode is a first class FX node: transforms, effects, translucency

Architecture

Notifications sun.swing.LightweightContent ‑ implemented in SwingNode: paint lock/unlock image buffer update focus grab/ungrab preferred size update cursor change

Limitations JViewPort: BLIT_SCROLL_MODE → BACKINGSTORE_SCROLL_MODE Why? Bypassing paint() call, overridden by JLF's content pane Looking for a better solution (getting somehow into RepaintManager) Mac OS X: -Dswing.volatileImageBufferEnabled=false Otherwise, performance drops significantly Why? Bliting bits from VolatileImage to the shared memory

Supported/planned functionality Platforms: Windows/Mac/Linux Key/Mouse/Wheel events Popups auto-hiding Custom Cursor change Dirty Regions Retina → JFX8 * Focus traverse-out → JFX8 * * Not a commitment. The targeted release number may be revised.

Future improvements Performance DnD Input methods A11y Modality / Z-order

More Information: “Embedding Swing Content in JavaFX Applications” http://docs.oracle.com/javafx/8/embed_swing/jfxpub-embed_swing.htm Relevant JavaOne talks (Swing and JavaFX): Practical Pros and Cons of Replacing Swing with JavaFX in Existing Applications [CON3530] – Tuesday 11:30 Moving from Swing to JavaFX [CON7852] – Thurs 12:30 There is a new tutorial that describes SwingNode on the Oracle doc site. Also, here are a couple of talks that deal with Swing and JavaFX interop.

Unified Threading Threading: the single biggest problem with Swing interop Forces asynchronous programming on the desktop Long considered to be “impossible to solve” AWT/Swing threading is complex Not dispatching events is equivalent to locking “Two locks, two threads? You can deadlock …” Probably the worst problem that FX and AWT/Swing interop has because it forces asynchronous programming. - it’s a hard problem because threads are hard - FX and AWT/Swing must talk to each other via runLater() and invokeLater() - note that there is no invokeAndWait() in FX not because one can’t be written, we just don’t want anyone waiting - if you wait, you can deadlock General Rule of Thumb: Never wait, runLater()

Threading: The Old Way JavaFX Operating System AWT Swing Event Queue This is a simple diagram of how threading works today. - each toolkit (FX and AWT/Swing) has their own event queue - each toolkit dispatches events to widgets from their own event thread - to talk to each other, the toolkits cross post messages in each other’s queues NOTE: This picture is a bit of a lie on the Mac where there is only a single operating system message queue per process but the principles are the same. The toolkits must cross post.

Threads in JavaFX and AWT/Swing JavaFX is apartment threaded The JavaFX platform GUI thread runs application code If application code waits, it correctly blocks the UI AWT is “free threaded” (not really …) The Event Dispatch Thread (EDT) runs application code The EDT is not the platform GUI thread The AWT platform GUI thread cross posts events to EDT Doesn’t wait (except for the next event from the operating system) Application code doesn’t run in the platform GUI thread JavaFX is aparment threaded and application code runs in the platform GUI thread. AWT is “free threaded” (not really, it’s kind of apartment threaded too but it’s not enforced) - application code runs the EDT which is not the platform GUI thread - AWT cross posts message from platform GUI thread

Threads in JavaFX and AWT/Swing Swing is “apartment threaded” (not enforced) The EDT runs application code (recall, it is not the GUI thread) Waiting in the EDT correctly block the Swing UI “So both JavaFX and Swing are apartment threaded and neither one should wait while running application code. In fact, applications are often coded not to wait by using workers etc. Hmmm …” Swing is apartment threaded but this is not enforced. - if you are not running the EDT thread, you need to use invokeLater() or invokeAndWait() - so both toolkits are apartment threaded (in different apartments) - both toolkits should not wait in their apartment thread

The Idea: Marry the Threads “Why not make the JavaFX platform GUI thread be the EDT thread for AWT/Swing? It is required that the EDT be non-GUI for AWT/Swing, but there is nothing that says it can’t be a GUI thread for someone else” This is the big idea. Why can’t the AWT/Swing EDT thread be the FX GUI thread?

Unified Threading: The New Way Operating System JavaFX Event Queue Event Queue AWT Swing This is the new diagram. - rather than cross posting to each others thread - AWT/Swing posts events to the FX GUI thread and FX dispatches both FX events and AWT/Swing events - the EDT and the FX GUI thread are the same thread so the two toolkits can talk directly

Unified Threading Still under construction for JavaFX 8.0 Early experiments are looking good Threading is complex so there may be dragons Enabled using a property Djavafx.embed.singleThread=true Disabled by default Unified threading is very new and threading is complicated, especially between platforms. - it is available in the JDK8 time frame but is not the default - please try it out Try it out and let us know how you get on!

Old and not so Exciting: Where we are today To fill a shape with an image. Use existing picture box, DO NOT delete and create new picture box. Right click on the shape. At the bottom of the submenu select “Format Shape” Select “Fill” at the top of the “Format Shape” dialog box. Select “Picture or Texture fill” from the options. And select “File” under the “Insert from” option. Navigate to the file you want to use and select “Insert” On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE Old and not so Exciting: Where we are today

JFXPanel: State of the Union Code is stable and functionality is reasonable Areas that need work: Opacity, Exit on Close, Performance, Modality, DnD, Layout, Popups auto-hiding, Focus Traversal JFXPanel has been around since FX 2.0 and the code is stable, however there are always areas that need work.

Opacity [P3] RT-31361 Opacity is ignored for scene contained in a JFXPanel (Since b94) [P3] JDK-8022512 JLightweightFrame: the content pane should be transparent Note: use -Dswing.jlf.contentPaneTransparent=true to make the SwingNode’s background transparent An undocumented property is currently provided which makes JLF fully transparent: -Dswing.jlf.contentPaneTransparent=true The main problem with opacity is mouse events processing on transparent spots. What can we do with it? 1. Make a property which allows for fully transparent background, like for SwingNode. MouseEvents will be captured, though. 2. Don't support transparency. 3. Implement mouse events filtering. See the dedicated mail thread: http://mail.openjdk.java.net/pipermail/awt-dev/2013-September/005392.html

Exit on Close [P3] RT-29926 IllegalStateException using JFXPanel in a netbeans window. [P4] RT-30536 JFXPanels are not reusable WORK AROUND: Platform.setImplictExit(false) and call Platform.exit() When the last window is closed, FX terminates. Can't restart FX thread: https://javafx-jira.kenai.com/browse/RT-23752. No plans for JFX8. A workaround is to use: Platform.setImplicitExit(false) and the explicitly call Platform.exit().

Performance [P3] RT-24278 JFXPanel with simple animation consumes entire CPU core (see PERF-326) [P3] RT-22567 Minor tweaks to FX/Swing painting Just need to get to these, we have lots of ideas <<1. BufferedImage tweaks (format, "manageable" status, etc.)      2. Avoid extra pixels copy from EmbeddedScene to JFXPanel      3. Doesn't call to EmbeddedScene from paintComponent(), if FX scene has not changed      4. Respect JFXPanel dirty regions, so we don't repaint the whole scene every time>> No any progress here so far.

Modalilty [P4] RT-26080 FileChooser: cannot show modal when using jfxpanel.scene.window as owner window WORK AROUND: Wrap JFXPanel in a JDialog. No work around for FileChooser This is a hot topic. I saw people complaining on it on the alias. There're two separate, but related problems actually. Modality itself and z-order. For a custom FX dialog there's a known workaround which is to wrap its content into JFXPanel wrapped into JDialog set as an owned dialog of the original Swing toplevel. However, for built-in dialogs like FileChooser this doesn't fit. Petr has been experimenting with z-order. He has a more or less working solution for Mac, and is working on Windows part. Probably we can demonstrate it on the BOF with an "experimental functionality" sticker. Concerning modality, there's no progress so far.

DnD [P3] RT-24337 Drag and drop window operations result in AssertionError with embedded JFXPanel components in NetBeans Platform Applications [P2] RT-24273 Mac: Dragboard don't work correctly with JFXPane [P3] RT-27014 Memory leak in EmbeddedScene.scenePaintListener or QuantumToolkit.dragSourceScene [P3] RT-24890 DRAG_DONE event doesn't come Just a number of critical issues

Layout [P4] RT-13858 JavaFX in Swing: no autosize on Propertychanges in Controls [P4] RT-27053 JFXPanel: race condition when setting scene Hoping to get to these for FX8 Should be fixed in JFX8.

Popups auto-hiding [P4] RT-19953 PopupWindow.autoHide does not work in Swing application (includes ContextMenu etc) Should be fixed in FX8. Work around in the JIRA (This works in SwingNode, btw). Should be fixed in JFX8. Currently, there's a workaround (https://javafx-jira.kenai.com/browse/RT-19953?focusedCommentId=315222&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-315222)

Focus traversal [P4] RT-10919 Provide possibility to traverse focus out of FX scene This one is bad but we aren’t able to get to this for FX8 No traverse-out support yet. Requires new functionality on FX side. (I doubt it's resolved in JFX8).

Discussion Topics: What should we do next? To fill a shape with an image. Use existing picture box, DO NOT delete and create new picture box. Right click on the shape. At the bottom of the submenu select “Format Shape” Select “Fill” at the top of the “Format Shape” dialog box. Select “Picture or Texture fill” from the options. And select “File” under the “Insert from” option. Navigate to the file you want to use and select “Insert” On the “Format” tab, in the Size group, click on “Crop to Fill” in the Crop tool and drag the image bounding box to the desired size DELETE THIS INSTRUCTION NOTE WHEN NOT IN USE Discussion Topics: What should we do next?

Discussion: What should we do next? Areas that need work: Retina support, Accessibility, Multi-touch, More conversion methods (SwingFXUtils, Color, Font etc.) … your input here … Multi-touch won't be supported. Need to document it.