1. VTT Technical Research Centre of Finland 3rd INTUITION Workshop “VR/VE & Industry – Challenges and Opportunities” Schwabenlandhalle, Fellbach / Stuttgart, Germany 30th of November – 1st of December 2006 BACKGROUND Complexity of an industrial plant is so huge that maintenance and operation personnel of the plant need advanced technology to access information related to the equipment maintained. Even though current maintenance and automation systems enable storage and browsing of equipment and process related data, they do not solve many of the common problems the maintenance personnel are facing, e.g.: how to locate the equipment needing maintenance, how to get intuitive and easily understood information about the plant floor equipment and their state, how to combine data from several sources without accessing many different applications with different user interfaces The problems are expected to come even more difficult in the future. Since owner/operators are outsourcing parts of their operations, new business opportunities occur in industrial services. While the change creates new business opportunities, it also creates new demands for software applications and services PLAMOS PROJECT The objective of Plamos project is to increase the productivity of the maintenance personnel by providing the users all the information needed in intuitive, augmented reality user interface. The plant model information is utilised in a mobile terminal where an augmented reality view is generated for a process operation and maintenance engineer. The mobile terminal is a tablet PC equipped with a camera and WLAN connection. Plamos creates new innovative solutions by combining several state-of-the art research approaches: plant model based standard information interfaces, visual browsing techniques, augmented reality and human technology interaction research. The Plamos user interfaces for mobile workers are specified on top of the standard maintenance application interfaces, e.g. OPC Unified Architecture and MIMOSA OpenO&M PLAMOS PROTOTYPE In the first Plamos prototype, the tracking solution is based on detecting visual tags attached to the plant environment. To enable greater flexibility, however, our final aim is to employ hybrid tracking methods with feature detection methods to determine the user’s orientation in the environment. Augmented Reality Plant Model Services for Mobile Maintenance Worker Pekka Siltanen, Tommi Karhela, Charles Woodward, Paula Savioja, Jani Jäppinen VTT Technical Research Centre of Finland The challenges are related to the application interoperability, as well as human system interfaces. While the machinery becomes increasingly complicated and maintenance is outsourced or carried out with smaller amount of onsite maintenance personnel, the need for intuitive user interfaces for locating the equipments and viewing the product and process information is increasing. Plant model information is normally distributed in several applications. In the Plamos prototype, the information is stored in one common data warehouse, the ProCore ontology modelling application. ProCore allows creation of links between different data models describing different views to the plant model, enabling information that has been traditionally represented in separate user interfaces to be shown in one graphical user interface. The system uses a small tablet PC (Sony VAIO with Windows XP) equipped with a video camera and WLAN connection. The augmented reality application is based on ARToolkit, which we have enhanced with e.g. adaptive thresholding, feature tracking and Kalman filtering. The application recognizes the visual tags and maps them to the plant model objects. Visual tags are also used for monitoring real-time process data of certain object. In the prototype process data was simulated with APROS process simulator, which has a standard OPC interface for remote access. Furthermore tags can be used for augmenting 3D model(s) of the plant over the video stream. The final integrated prototype will combine the information stored in maintenance system with AR and 3D views all in same user interface. Using Plamos the user can also report problems noticed while performing maintenance tasks. Plamos Augmented Reality GUI OPCSVGVRML MIMOSA OPEN O&M Automation system/ Process simulator P&ID Schemas 3D Models Maintenance information ProCore