Pose Estimation TUE. Kim Kyungkoo Active Grasp

2 Introduction Pose Estimation –Object modeling with features –Real-time pose estimation Demo Future works Contents

Introduction Importance of object recognition and pose estimation 3

Pose Estimation Problem Definition –Robot knows The target object to grasp The corresponded 3D model The grasp point on a 3D model –BUT! Do not know The grasp point in real-environment 4 Orientation matching between an object and a 3D model is needed

Pose Estimation System overview –Object modeling –Automatic pose estimation 5 Stereo Camera Live video TrackingReconstruction Partial model Model features Transformation Stereo Camera Live video Feature matching Pose estimation 3D model of an object

Object Modeling with features Object Modeling process 6 2D image 3D depth Image Disparit y Bi-layer Segmentation 2D image 3D depth Image Object Segmentation Object depth image SURF Feature Tracking 2D image 3D depth Image Merged Object Depth image Merging Disparity Image Homogeneous Matrix Calculation Merged Foreground Depth image Merged Image Set Captured Image Set Accumulated Image Set Depth Image Reconstruction

Object Modeling with features Object feature list creation during modeling process –Features Using SURF algorithm to extract features Each feature consist of a 3D coordinate and a descriptor –Storing features extracted from object region of each frame As the system extracts features from each image, it accumulates the features with a previous feature list –It stores all features for the first image in image stream 7 A 3D Feature SURF Feature Match Updated 3D Feature list Transformed 3D Feature list Matched? YES NO Add feature descriptor into same ID Create new ID for corresponding points

Feature list creation on an object Example of feature list 8

Real Time Pose Estimation Feature matching between feature list of an object and features of current image –Using SURF feature extraction and matching algorithm –Each feature consist of a 3D coordinate and a descriptor –Acquisition of 3D corresponding points Transformation –The 3D model of an object is transformed to fit a current image using 3D corresponding points –Method? 9

Pose Estimation of current view Transformation of the 3D model for pose estimation –Using three corresponding points –Calculate the best transformation matrix with three corresponding points using RANSAC algorithm 10

Pose Estimation of current view Transformation of the 3D model for pose estimation 11 H

Pose Estimation of current view Transformation of the 3D model for pose estimation 12 3 쌍의 corresponding points 중 random 하게 한 쌍을 선택하여 선택된 각 점을 3 차원 공간상 0,0,0 으로 이동 T1T1 T2T2

Pose Estimation of current view Transformation of the 3D model for pose estimation 13 남은 2 쌍의 corresponding points 중 한 쌍을 선택하여 그 점이 같은 축 위에 존재 하도록 회전 R1R1

Pose Estimation of current view Transformation of the 3D model for pose estimation 14 같은 축 위로 회전된 corresponding point 를 기준으로 scaling

Pose Estimation of current view Transformation of the 3D model for pose estimation 15 마지막 남은 corresponding point 를 다른 쪽에 맞도록 회전

Pose Estimation of current view Transformation of the 3D model for pose estimation 1.Choose three corresponding points randomly 2.Calculate a transformation matrix 3.Transform all the corresponding point of model using the transformation matrix 4.Sum the distance between each corresponding point 5.Repeat 1 st to 4 th process 6.Select the transformation matrix which contains minimum distance summation value 7.Transform all the point of an object model using the inverse matrix of the selected transformation matrix in 6 th process 16

Demo Modeling process 17

Demo Pose estimation 18

Future Works Accuracy Transformation Feature list 19