1. AR Based NPP Maintenance M. Khalaquzzaman NIC Lab KAIST 25 Feb 2008

2. Outlines  Definition of AR  Marker based AR  Applications of AR in NPP  Major Hindrance of AR implementation in NPP  Conclusions

3. Definition of Augmented Reality Virtual Environments (VE): Completely replaces the real world Augmented Reality (AR): User sees real environment; combines virtual with real - Supplements reality, instead of completely replacing it

4. Example Augmented Reality Automobile Engine

5. Marker Based AR Process

6. Systems of Augmented Reality Display –Head mounted –Handheld Viewpoint Tracking –Magnetic –Ultrasonic –Computer Vision Image Generation

7. Extension of AR with WSN

8. Application of AR in NPP Maintenance Support Activities 1.Navigating workers to the workplace 2.Indicating dangerous locations 3.Making invisible visible Next Place Danger ! In operation Main Steam Line Pressure 25.0 MPa

9. Major Obstacles to Application of AR to Maintenance Work Support  Huge area with a large numbers of machineries  Workers need to move in the large area during maintenance work  Legacy AR is not capable to move in a wide area  accuracy of tracking for maintenance work must must be high compare to the legacy AR application  workers need to perform many complex task so that they need to move so fast and to see their feet  they must be careful about their not to hit their heads on protuberances such as pipes and valves  the HMD limit their view, and their weight should as light as possible otherwise that will fatigue workers

10. Researches to enable AR to be applied to Maintenance Work Support According to the research of Hirotake Ishii and his fellows in Kyoto University, many issues must be resolved before introducing the AR into maintenance…  Developing of Tracking Method using Markers ( i.e. line Marker and circular Marker)  Marker arrangement optimization  Comparison of HMD

11. Potential Applications Augmented Reality  Among the tracking technologies proposed in the previous studies, Marker- based tracking seems to be most appropriate for the use in nuclear power plants. Available markers are: 1.Square marker 2.Circular marker 3.Line marker 4.Nested marker

12. Potential Applications Augmented Reality Square Marker: it is most popular marker but larger size is needed to recognize from a distant places; Circular Marker: Line Marker: Nested marker

13. Hybrid Tracking Method Gyro-sensors are used to measure the rotating rate of objects. The information provided by gyros can be used to trigger automobile airbags during rollover, improve the accuracy and reliability of GPS/navigation systems, and stabilize moving platforms such as automobiles, airplanes, robots, antennas, and industrial equipment A hybrid tracking method using a multi-camera unit and gyro-sensor, which can reduce umber off markers. One option to reduce the number of markers is to use wide-angle camera. However the image resolution decreases and distortion increases with the increase of view angle.

14. Using the multi-camera unit described above, the number of the line markers that must be pasted in the environment can be decreased at a certain level. But when a part of the line marker is hidden by a user’s body or tools used for maintenance, the tracking will not be able to continue. A hybrid tracking method that combines a gyro-sensor and the line marker. From the gyro-sensor, only the orientation is obtainable at all times, but the output is not so accurate and the cumulative error will increase as time passes. Therefore, the gyro- sensor is useful only for a very short period of time. Hybrid tracking :

15. Extension of Viewing Angle with Multi- Camera Unit Extension of View Angle A multi-camera unit with three cameras view angle 40 degrees each multi-camera can capture three-times wider-angle images than a single camera. increase the angle of view by three-times algorithm is different from single camera system Cases: 1. Only one marker can be captured by a single camera 2. One camera can capture more than one marker 3. No marker can be captured by a camera..

16. Head tracking with a Camera in the back side Camera 2 camera1 Line marker Base for placing marker

17. Head tracking with a Camera from the back side  Camera is hung over a pool attached at the back  Marker is inserted on the HMD Another arrangement :

18. Head tracking with a Camera in the back side Camera 2 camera1 seen in addition to other stuffs

19. Modeling of Nuclear Power Plant

20. Modeling With AR

21. AR of Markers on the Desk 3D cut-away diagram

22. Arrangement of Markers on the Floor Floor of a room Floor with marker

23. Conclusions  The use of fiducial markers is unavoidable because nuclear power plants necessitate highly reliable tracking;  a hybrid tracking method that combines a gyro-sensor and the line marker has been employed. From the gyro-sensor, only the orientation is obtainable at all times, but the output is not so accurate and the cumulative error will increase as time passes.

24. References: 1.A Nested Marker for Augmented Reality, Keisuke TATENO, Itaru KITAHARA, and Yuichi OHTA; Graduate School of Systems and Information Engineering, University of Tsukuba; 2. Augmented Reality Applications for Nuclear Power Plant Maintenance Work ; Hirotake Ishii 1, Zhiqiang Bian 1, Hidenori Fujino 1, Tomoki Sekiyama 1, Toshinori Nakai 1, Akihisa Okamoto 1, Hiroshi Shimoda 1, Masanori Izumi 2, Yoshinori Kanehira 2 and Yoshitsugu Morishita 2 ; Graduate School of Energy Science, Kyoto University, Kyoto, Japan. 3. Chipcentre, Test and Measurement ; http://archive.chipcenter.com/TestandMeasurement/products_401-500/prod410.html http://archive.chipcenter.com/TestandMeasurement/products_401-500/prod410.html

25. Thank you