Tobias Kohoutek Institute of Geodesy and Photogrammetry Geodetic Metrology and Engineering Geodesy ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Second Baltic Swiss Geodetic Science Week September 2007, Neringa, Lithuania

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Monitoring! Why? - monotone and physical stressful jobs - autonome moveable machines using of industrial robots - safety shut-off mats, light barriers, camera systems

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Used Components -LEGO® Mindstorms™ RCX 2.1 -SwissRanger® SR-3000 (CSEM/MESA)

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Range Imaging -combined CMOS/CCD-technology -parallel recording of local brightness and distance map -distance measured by time-of-flight (TOF) principle for every pixel -map can be acquired by detected phase delay -signal sampling at four points to determine offset I, amplitude A and phase φ Swissranger® SR-3000

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania 3D-Image Processing -managing of a 3D-image block -image analysing for segmentation and object tracking -accuracy and reliability -automatic real time collision prevention objectives

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania 3D-Image Processing 3D-image-data by SR-3000 amplitude image coloured range image

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Image Analysing two different models – for robot installation: robot detection by background image (without robot) and following image with robot → subtract images and differences become visible immediately – for preexisting robot: detection without background image

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Image Analysing detection by background image

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Image Analysing image avaraging - noise reduction by average over N images - noise level reduced by 1/√N compared to single image edge detection - OpenCV-function cvCanny() for binary image close gaps in robot contour fill robot structure detection without background image

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Image Analysing detection without background image

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Image Processing feature detection real time motion analysis for 3D-image-data -> result: working space of robot image monitoring for other changes - reduction of monitored space - minimize computing time - other workers/machines detected in image, don‘t influence the robot

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Image Processing feature detection

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Image Processing image monitoring

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Motion Analysis based on three special features virtual box around robot, width = 10 pixel static security zone

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Motion Analysis all edge pixels which edge/corner? -> neighborhood operation zone boarder by virtual feature moving from image centre to image frame dynamic security zone

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Automatic Real Time Monitoring security zone (red line) is monitored real time 25 frames/second security stop by entering object security zone

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Results robot detection by image analysis in range image automatic real time object tracking by Optical Flow with subpixel accuracy in amplitude image security zone moves with robot collision prevention in real time (amplitude and range image) in three distances Problems: error-prone algorithms in C++ program high reflexions at robot material little sensor size -> robot in image centre

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Results

ANALYSIS AND PROCESSING OF 3D-IMAGE-DATA FOR ROBOT MONITORING Tobias Kohoutek Second Baltic Swiss Geodetic Science Week 10 – 14 September 2007, Neringa, Lithuania Thank you for your attention! Please, feel free to ask questions.