1. DataNaut, LLC http://www.datanaut.com 1 Building Cross-Platform Accessible Applications Using Qt Technical Overview September 2012

2. DataNaut, LLC http://www.datanaut.com 2 Introduction Overview – Presentation covers modern architectural patterns such as Model View ViewModel (MVVM) combined with the latest cross-platform technologies including Qt and Webkit to create interactive and accessible applications. – Using Qt and Webkit offer significant advantages when targeting multiple platforms however accessibility is an issue. – This presentation covers what steps you can take to leverage these powerful tools and technologies and meet accessibility requirements Review usage example Accessibility in Qt Approach Tools and Technologies

3. DataNaut, LLC http://www.datanaut.com 3 Working Example Review working example using this technical approach Learning Ally Audiobook Manager – Used by Learning Ally subscribers to manage books that they can access – Users download books which are synchronized with iTunes, Windows Media or DAISY device – Users interact with a bookshelf which presents the books in an accessible manner – Application has HTML User Interface hosted within a cross-platform Qt application – Find out more at http://www.learningally.orghttp://www.learningally.org Key goals – Desktop application that supports Windows and Mac platforms – Support for screen readers – JAWS, Window-Eyes, NVDA, native – Leverage Webkit and Qt – Implement Model View ModelView pattern

4. DataNaut, LLC http://www.datanaut.com 4

5. 5 Application Approach Overview Application implemented as a cross-platform thick client with rich user interface and advanced accessibility support Cross platform support is provided by Qt framework with UI built using the web technologies (HTML/CSS/JavaScript) embedded into the thick client using WebKit control By using the WebKit control, hosting web assets helps to facilitate building the UI using the standard web technologies while still providing thick client advantages such as file system access, advanced networking capabilities, accessibility and other lower level OS capabilities Why? – Not all applications will work in the web browser. HTML5 goes a long way but some applications need more – Webkit provides the capability to implement a modern web-based user interface – Qt does not provide complete accessibility support, need approach to support it in a cost effective manner

6. DataNaut, LLC http://www.datanaut.com 6 Application Approach Issues with Qt and Accessibility Qt has limited support for accessibility on Windows and Mac OS. The majority of stock Qt controls do not support accessibility Qt has it’s own implementation of Webkit called QWebKit which has it’s own accessibility issues An approach is to implement custom control with accessibility support using interfaces provided by Qt, e.g. an accessible wrapper around QWebKit Official Qt forums and docs advise to override standard controls with your own implementation of accessibility The best approach is that the Qt community adds support for IAccessible and IAccessible2 but this is not in the pipeline

7. DataNaut, LLC http://www.datanaut.com 7 Application Approach Adding Accessibility to QWebkit Implement custom wrapper around standard QWebKit control that supports accessibility Wrapper receives messages with active control and its state from HTML (via JavaScript) in QWebKit and sends this information to the OS This works on Windows but there are issues on Mac – By default the OS reads accessibility details once for each control and doesn’t support dynamic changes of accessibility details – Since this approach changes the accessibility details on the fly it’s a show-stopper – Requires modifications to alter how Qt processes accessibility-related changes to have the ability to send control changes to the system Benefits – End result is an accessibility façade that can be primarily controlled from HTML and JavaScript – It’s very flexible. You can control navigation between HTML controls, control state and accessible details – Overall the code is easier to maintain, works better with web designers and development team

8. DataNaut, LLC http://www.datanaut.com 8 Application Approach Using JavaScript Handle accessibility in two parts - – Navigation between controls – Manage what is announced when on a control Most of the work is done in JavaScript and Qt is used as a proxy to send updated controls state and hints to the OS Generate navigation map tables in JavaScript for all accessible controls in the web page Add a custom attribute to HTML tag for accessibility specifics including what’s announced to the user

9. DataNaut, LLC http://www.datanaut.com 9 Application Approach Accessibility Facade Each web page has a navigation map that defines accessibility properties in JavaScript:

10. DataNaut, LLC http://www.datanaut.com 10 Application Approach Accessibility Facade Continued To support what is announced by the reader there are special html attributes for controls to describe the type and voice hint When the user mouses over or navigates to a control, the control sends a message to Qt with control type, position, and accessible text. Qt forwards this info to the system to make it available to the screen reader

11. DataNaut, LLC http://www.datanaut.com 11 Application Approach Accessibility Facade Continued Qt support -

12. DataNaut, LLC http://www.datanaut.com 12 Future Developments Support for IAccessible2 in Qt Qt accessibility is based on IAccessible interface however it’s not complete The problem is that Qt does not send enough information to the screen readers to properly announce the controls The appropriate course of action is to implement IAccessible2 interface for Qt controls. However this is a significant effort and not a high priority in the Qt development pipeline DataNaut has performed R&D to add IA2 for several controls Our goal is to finalize IA2 support for these controls and lobby Qt community for inclusion in the current build

13. DataNaut, LLC http://www.datanaut.com 13 Tools, Technologies, Resources Qt – http://qt.nokia.com/ Webkit – http://www.webkit.org/ Model View ViewModel – http://en.wikipedia.org/wiki/Model_View_ViewModel Knockout MVVM Library - http://knockoutjs.com/ Open Accessibility Group - http://www.linuxfoundation.org/collaborate/workgroups/accessibility

14. DataNaut, LLC http://www.datanaut.com 14 Questions? Contact Mark Snuffin at snuffin@datanaut.com