1. Kim, Sung-Ho School of Computer, Information and Communication Engineering Sangji Univ.

2. 증강현실의 개요 ARToolKit 의 특징 What does ARToolKit do? How does ARToolKit work? How to install & setting up it? Setting up ARToolKit Camera Calibration Coordinate System ARToolKit Tutorials References 차 례차 례차 례차 례 차 례

3. 가상 현실 (Virtual Reality) 의 한 분야로 실제 환경에 가상 사물이나 정보를 합 성하여 원래의 환경에 존재하는 사물처럼 보이도록 하는 컴퓨터 그래픽스 기 법으로,“ 증대, 확대 ” 라는 의미를 담고 있는 증강 (Augmentation) 이라는 단어와 눈앞에 펼쳐진 실제 현실 (Reality) 이라는 단어가 합쳐진 합성어 어떤 특정한 환경이나 상황을 컴퓨터로 만들어서 그것을 사용하는 사람이 마 치 실제 주변 상황이나 환경과 상호작용을 하고 있는 것처럼 만들어 주는 인 간과 컴퓨터 사이의 인터페이스 (Interface) – 인공 현실 (Artificial Reality), 사이버 공간 (Cyber Space) – 가상 세계 (Virtual Worlds), 가상 환경 (Virtual Environment) – 합성 환경 (Synthetic Environment), 인공 환경 (Artificial Environment) – 시뮬레이션 기술 (Simulation Technology) 가상현실 : 현실 세계를 대체하여 창조된 새로운 환경 증강현실 : 현실 세계에 가상의 정보 ( 객체 ) 를 중첩함 (Mixing) 으로써 현실 세계 를 보충하여 사용자에게 보여 주는 환경 – 현실의 이미지와 가상의 이미지를 결합한 것 – 실시간으로 Interaction 이 가능한 것 – 3D 공간 안에 놓인 것 증강현실의 개요 증강현실의 개요

4. 증강현실의 요소 기술 – Marker Detection Technology 기준 좌표 역할을 하는 사각형의 마커를 인식하고 그 마커를 기준점 으로 하는 기술 – Markerless Tracking Technology Feature based Tracking 영상 내에서 특징 점들을 추출하고 이 특징 점들을 기반으로 좌표 계 를 추출 특징 점을 이용하기 때문에 사용자가 정의한 특정 패턴을 마커로 이 용할 수도 있다. 증강현실 기술에 있어서 가장 난이도 있고 점차 더 중요한 기술이 될 것으로 평가 증강현실의 개요 ( 계속 ) 증강현실의 개요 (계속)

5. 단일 카메라의 위치 / 방향을 추적한다 검은 사각형을 사용하는 추적코드는 간단하다 사각형의 Marker Pattern 을 사용한다 간편하게 카메라 코드를 보정 할 수 있다 빠른 실시간 어플리케이션이다 무료로 배포되는 오픈 소스이다 PC CAM( 화상카메라 ), Visual studio 6.0 이상, DirectX 9.0b 이상 ARToolKit 의 특징 ARToolKit의 특징

6. ARToolKit 의 Architecture ARToolKit의 Architecture

7. ARToolKit 의 Structure ARToolKit의 Structure

8. ARToolKit - Free tracking library – Software library for building AR applications based on OpenGL – Calculate the real camera position and orientation relative to physical markers in real time – 3D VRML support – Software Library for vision-based AR applications Open Source(C language), multi-platform(SGI IRIX, PC Linux, PC Windows) – Overlays 3D virtual objects on real markers Uses single tracking marker Determines camera pose information (6 DOF) – Includes utilities for marker-based interaction – ARToolKit Website http://www.hitl.washington.edu/artoolkit/ Limitations – Monocular camera setup 을 사용하므로 3D 측정이 불가능함 – Lighting condition 과 Marker 의 재질에 민감하게 반응할 수 있음 What does ARToolKit do ? What does ARToolKit do ?

9. Principle of ARToolKit – http://www.hitl.washington.edu/artoolkit/documentation/userarwork.htm http://www.hitl.washington.edu/artoolkit/documentation/userarwork.htm How does ARToolKit work? How does ARToolKit work?

10. static void mainLoop(void) { static int contF = 0; ARUint8 *dataPtr; ARMarkerInfo *marker_info; int marker_num; int j, k; /* grab a vide frame */ if( (dataPtr = (ARUint8 *)arVideoGetImage()) == NULL ) { arUtilSleep(2); return; } if( count == 0 ) arUtilTimerReset(); count++; argDrawMode2D();... How does ARToolKit work? (con’t) How does ARToolKit work? (con’t)

11. ..... /* grab a vide frame */ if( (dataPtr = (ARUint8 *)arVideoGetImage()) == NULL ) { arUtilSleep(2); return; } if( count == 0 ) arUtilTimerReset(); count++; argDrawMode2D(); argDispImage( dataPtr, 0,0 ); /* detect the markers in the video frame */ if( arDetectMarker(dataPtr, thresh, &marker_info, &marker_num) < 0 ) { cleanup(); exit(0); } arVideoCapNext();..... How does ARToolKit work? (con’t) How does ARToolKit work? (con’t)

12. .... if( arDetectMarker(dataPtr, thresh, &marker_info, &marker_num) < 0 ) { cleanup(); exit(0); } arVideoCapNext(); /* check for object visibility */ k = -1; for( j = 0; j < marker_num; j++ ) { if( patt_id == marker_info[j].id ) { if( k == -1 ) k = j; else if( marker_info[k].cf < marker_info[j].cf ) k = j; } if( k == -1 ) {.... How does ARToolKit work? (con’t) How does ARToolKit work? (con’t)

13. .... /* check for object visibility */.... /* get the transformation between the marker and the real camera */ arGetTransMat(&marker_info[k], patt_center, patt_width, patt_trans); draw(); argSwapBuffers(); } How does ARToolKit work? (con’t) How does ARToolKit work? (con’t)

14. static void draw( void ) {.... /* load the camera transformation matrix */ argConvGlpara(patt_trans, gl_para); glMatrixMode(GL_MODELVIEW); glLoadMatrixd( gl_para ); glEnable(GL_LIGHTING); glEnable(GL_LIGHT0); glLightfv(GL_LIGHT0, GL_POSITION, light_position); glLightfv(GL_LIGHT0, GL_AMBIENT, ambi); glLightfv(GL_LIGHT0, GL_DIFFUSE, lightZeroColor); glMaterialfv(GL_FRONT, GL_SPECULAR, mat_flash); glMaterialfv(GL_FRONT, GL_SHININESS, mat_flash_shiny); glMaterialfv(GL_FRONT, GL_AMBIENT, mat_ambient); glMatrixMode(GL_MODELVIEW); //glTranslatef( 0.0, 0.0, 25.0 ); glTranslatef( 0, 0, 25.0 ); glutSolidCube(50.0); glDisable( GL_LIGHTING ); glDisable( GL_DEPTH_TEST ); How does ARToolKit work? (con’t) How does ARToolKit work? (con’t)

15. http://www.hitl.washington.edu/artoolkit/documentation/userarwork.htm – Live video image 를 binary image 로 변환 (lighting threshold value 사용 ) – binary image 에서 모든 square region 탐색 – 각 square region 내의 pattern 을 capture 하여 pre-trained pattern – template 들과 match 를 시도하여 실제 marker 인지를 결정 – 일단 한 marker 를 찾으면, 알려진 square size 와 pattern orientation 을 – 사용하여 (marker 에 상대적인 ) real video camera 의 position 을 계산함 – 실제 camera 를 계산하면, virtual object 들을 image 에 draw 할 수 있음 How does ARToolKit work? (continued..) How does ARToolKit work? (continued..)

16. Multiplateform library (Windows, Mac OS, Linux) For Windows – Download OpenGL – GLUT GLUT DSVL(DirectShow Video Processing Library) – Communication with the camera driver Communication with the camera driver ARToolkit library OpenVRML Libpng / jpeg Libpngjpeg DXSDK msvcr71d.dll, msvcp71d.dll msvcr71d.dllmsvcp71d.dll – Building library – Visual Studio 6 & Visual Studio.NET 2003 How to install & setting up it ? How to install & setting up it ?

17. http://www.hitl.washington.edu/artoolkit/documentation/usersetup.htm http://abekdh.tistory.com/9 http://www.mironae.com/1008 http://niceyth.springnote.com/pages/409596 Compile SimpleVRML with OpenVRML Setting up ARToolKit Setting up ARToolKit

18. Calibration – 특정조건에서 측정기기, 표준물질, 척도 또는 측정체계 등에 의하여 결정 된 값을 표준에 의하여 결정된 값 사이의 관계로 확정하는 일련의 작업을 말한다 Camera Calibration – 카메라의 정보 ( 내부변수 [intrinsic parameter] 와 외부변수 [extrinsic parameter]) 를 알아내는 작업으로서, 실 세계의 장면이 카메라를 통해 픽셀 로 변환되면 왜곡이 발생하는데 이러한 것들을 보정하는 작업을 말한다.. – 카메라의 정보 : ARToolKit/bin/Data/WDM_camera_flipV.xml http://www.hitl.washington.edu/artoolkit/documentation/usercalibration.htm – Two Step Calibration – One Step Calibration Camera Calibration Camera Calibration

19. http://www.hitl.washington.edu/artoolkit/documentation/cs.htm – Computer Vision Coordinate System – Rendering Coordinate System Coordinate Systems Coordinate Systems

20. Tutorial 1 : Tracking Stability – http://www.hitl.washington.edu/artoolkit/documentation/tutorialhistory.htm http://www.hitl.washington.edu/artoolkit/documentation/tutorialhistory.htm Tutorial 2 : Camera and Marker Relationships – http://www.hitl.washington.edu/artoolkit/documentation/tutorialcamera.htm http://www.hitl.washington.edu/artoolkit/documentation/tutorialcamera.htm Tutorial 3 : Multi-Marker Tracking – http://www.hitl.washington.edu/artoolkit/documentation/tutorialmulti.htm http://www.hitl.washington.edu/artoolkit/documentation/tutorialmulti.htm Camera Calibration & Marker Generation – http://gekko.tistory.com/category/prog/AR http://gekko.tistory.com/category/prog/AR ARToolkit for Dummies – http://isa.umh.es/vr2/euron06/doc/labs/ARToolkit_for_Dummies.pdf http://isa.umh.es/vr2/euron06/doc/labs/ARToolkit_for_Dummies.pdf How to import a new VRML model How to make your own marker How to make your own AR applications ARToolKit Tutorials ARToolKit Tutorials

21. \ARToolkit\bin\Data\object_data_vrml for simpleVRML.exe Conceptual overview of the coherence between pattern, data and image files Expanding the S/W Expanding the S/W

22. The patterns to be recognized and their associated models are specified in the Data/object_data_vrml file This means that the VRML content associated with the pattern "patt.hiro" is specified in the file "Wrl/bud_B.dat" –#pattern 1 –VRML Wrl/bud_B.dat –Data/patt.hiro –80.0 –0.0 0.0 The file line specifies the actual VRML file to be loaded, while the next three lines set the translation, rotation and scale of the VRML file from the pattern original –Wrl/bud_B.wrl –0.0 0.0 0.0 # Translation - x,y,z from center of tracking pattern –0.0 0.0 0.0 0.0 # Rotation angle + axis, eg 90.0 1.0 0.0 0.0 –2.5 2.5 2.5 # Scale x,y,z Expanding the S/W (Continued…) Expanding the S/W (Continued…)

23. 1. Copy your_file.wrl into the Wrl directory 2. Make a your_file.dat file and associate it with your_file.wrl by typing Wrl/your_file.wrl on the first line. –You may take a copy of one the existing.dat files and edit it to fit your wrl file –Place it in the Wrl directory 3. Make a new marker or use one that is not yet in use. 4. Edit the Data/object_data_vrml file by adding a new paragraph: –#pattern 3 –VRML Wrl/your_file.dat –Data/patt.sample1 -- Alternatively your new marker –80.0 –0.0 0.0 How to import a new VRML model How to import a new VRML model

24. To make it simple you can use the blankPatt.gif located in the patterns folder of the ARToolKit installation Draw your new pattern inside the black square The next thing you have to do is start the mk_patt.exe file located in the bin folder. The file requires an input for the camera. You can use "data/camera_para.dat". This is the default setting for the camera. Position the camera right over the marker. You should be able to see a red and green square around the pattern. This means that ARToolKit has found your marker. Rotate the camera until the red corner is in the upper left How to make your own marker How to make your own marker

25. Left click your mouse inside the viewing frame. Switch to the command window and type the name of your pattern same as “data/patt.mysamplepattern” To use your new pattern you have to edit object_data_vrml It should look something like this: –#the number of patterns to be recognized –1 –#pattern 1 –VRML Wrl/bud_B.dat –Data/patt.hiro –80.0 –0.0 0.0 How to make your own marker (Continued…) How to make your own marker (Continued…)

26. Explanation: 1. When you add a new set the new number of patters to be recognized: –#The number of patterns to be recognized –2 2. Add a new pattern number: –#pattern 2 3. Insert reference to the DAT file of the VRML model to overlay the pattern: –VRML Wrl/maggie.dat 4. Insert reference to your new pattern. You can use any type of filename –Myfolder/mysamplepattern 5. Adjust the height of the overlayed object How to make your own marker (Continued…) How to make your own marker (Continued…)

27. Explanation (Continued…): 6. X and Y coordinates relative to the center of the pattern: –10.0 5.0 Then you should end up with something like this: –#The number of patterns to be recognized –2 –#pattern 1 –VRML Wrl/bud_B.dat –Data/patt.hiro –80.0 –0.0 0.0 –#pattern 2 –VRML Wrl/maggie.dat –Myfolder/mysamplepattern –105.0 –10.0 5.0 How to make your own marker (Continued…) How to make your own marker (Continued…)

28. www.se.rit.edu/~jrv/research/ar/introduction.html www.cs.unc.edu/Research/ProjectIndex/GraphicsImage.html www.itk.ntnu.no/KIKS/demo/ www.csl.sony.co.jp/project/ar/ref.html www.howstuffworks.com/augmented-reality.htm Interesting Links Interesting Links

29. http://www.hitl.washington.edu/artoolkit/documentation/ http://jhseo.tistory.com/entry/ARToolKit-VRML-Programming http://isa.umh.es/vr2/euron06/doc/labs/ARToolkit_for_Dummies.pdf http://jhseo.tistory.com/135 http://makerfaireri.crowdvine.com/attachments/0000/4771/ARToolkit_Blender.pdf http://old.uvr.gist.ac.kr/wlee/web/papers/2004/KHCI04_kkim_wlee.pdf http://www.medien.ifi.lmu.de/lehre/ss09/ar/Installation_vom_ARToolKit.pdf http://www.hitl.washington.edu/artoolkit/Papers http://gekko.tistory.com/45 http://www.artoolworks.com/ARToolKit_Professional.html http://www.artoolworks.com/support/library/ARToolKit_Professional http://www.pragprog.com/titles/cfar/augmented-reality http://jpyun56.wordpress.com/2010/01/18/augmented-reality- application%EB%93%A4/ http://www.se.rit.edu/~jrv/research/ar/ http://makerfaireri.crowdvine.com/attachments/0000/4771/ARToolkit_Blender.pdf http://cafe.naver.com/opencvReferences References