1 DTI/EPSRC 7 th June 2005 Reacting to HCI Devices: Initial Work Using Resource Ontologies with RAVE Dr. Ian Grimstead Richard Potter BSc(Hons)

2 DTI/EPSRC 7 th June 2005 Presentation Structure Aims and Objectives Background to Ontologies Background to RAVE Initial extension of RAVE Demonstration Questions and Answers.

3 DTI/EPSRC 7 th June 2005 Aims To enhance RAVE Extend resource-awareness Include user input devices To prototype ontological approach In an existing system.

4 DTI/EPSRC 7 th June 2005 Objectives Implement Widgets Enable Widgets to request interaction support Enable RAVE to access Ontology RAVE widgets announce interaction style required Ontology finds best fit Resource interfaced with RAVE for information exchange.

5 DTI/EPSRC 7 th June 2005 Background to Ontologies Ontology: Formal collection of inter-referenced taxonomies Taxonomy – a hierarchical classification Resources described using 6 taxonomies: Hardware User actions Sensory experiences Software interfaces Software interface items Variables.

6 DTI/EPSRC 7 th June 2005 Resource description

7 DTI/EPSRC 7 th June 2005 Categorisation Each taxonomy categorises information, for example the user interaction taxonomy:

8 DTI/EPSRC 7 th June 2005 Libraries and APIs Sesame (RDF/RDFS repository) OwlIm (Sesame ontology plugin) Sesame-to-Jena Jena (Ontology API and inference engine) OWL/RDF/RDFS (Ontology implementation languages).

9 DTI/EPSRC 7 th June 2005 Background to RAVE RAVE – Resource-Aware Visualization Environment Supports collaborative visualization Static datasets, or Real-time feed from remote process Remote process can be instrumented RAVE can then steer a simulation.

10 DTI/EPSRC 7 th June 2005 Data Distribution First component: Data Server Acts as a distribution point & interpreter Understands many types of data Uses Java3D+Xj3D as importer Data Server Internet or remote machine Visualization Data RAVE Client RAVE Client RAVE Client Raw Data

11 DTI/EPSRC 7 th June 2005 Displaying Data Second component: Active RAVE Client Active – facilities to draw on its own Accepts feed from Data Server Presents images of data to user Visualization Data Server Active RAVE Client Visual drawn on local machine

12 DTI/EPSRC 7 th June 2005 Computational Steering Independent simulation can supply Data Server Simulation code instrumented Transmits scene creation and updates to Data Server Data Server reflects updates Multiple clients can view live simulation Client interact with widgets Steer simulation.

13 DTI/EPSRC 7 th June 2005 Summary Data Server reads data and distributes Active Client renders locally Data Server can link to live simulation All resources shared where possible Uses Java to support (most) platforms Also: Thin Client (PDA) renders via Render Server But thats another story.

14 DTI/EPSRC 7 th June 2005 Resource support Semantic reasoning over the ontology: Appropriate resources can be chosen Ontology also describes: Each resources available software interfaces Joins visualization to selected hardware Supports intuitive interaction Selects most appropriate resource.

15 DTI/EPSRC 7 th June 2005 Resource support

16 DTI/EPSRC 7 th June 2005 Initial Extension to RAVE Database to store ontology RAVE extended with new Widget type Issues: Lack of driver support! Only available: mouse / mouse with wheel Shows proof of concept.

17 DTI/EPSRC 7 th June 2005 Demonstration We will show: RAVE interacting with a molecular dynamics simulation RAVE discovering available input devices Ontology support automatically selecting available devices RAVE being unaware of available input devices.

18 DTI/EPSRC 7 th June 2005 Questions and Answers Questions… …and possibly answers?