1. Optoranger: a 3D pattern matching method for bin picking applications
G. Sansoni, P. Bellandi, F. Leoni and F. Docchio
Laboratory of Optoelectronics, Department of Information Engineering
School of Engineering, University of Brescia
Buongiorno a tutti

2. Aim of the activity
To design and implement a 3D vision system for bin-picking applications
To test a full 3D approach based on template matching

3. Robot bin picking Typical problem in industrial plants
Need to increment the level of automatization of the production line: flexibility and efficiency determine the overall performance
Robots are blind in nature: vision can be of great help to take advantage of their high repetability.
The work area is intrinsically three-dimensional
3D vision systems are naturally suited for accomplishing this task
Acquisition of 3D data
Extraction of information to provide the robot the pose and the orientation of each object, for a correct pick-up
Extraction based on
Segmentation
Object recognition

4. 3D cloud segmentation
Aimed at isolating the objects from the background
Very complex problem
A number of techniques available
The state-of-the-art shows methods based on:
the analysis of depth discontinuities of the 3D map
Region growing and curvature estimation
Combination of 2D images and 3D point clouds
Use of colour information
In this work:
the analysis of the 3D map is based on the computation of the euclidean distance among points;
Object segmentation is performed by evaluating the point cloud local density;

5. Object recognition (Matching)
Aimed at recocgnizing ‘what is what’
Very complex problem
A number of techniques available
The state-of-the-art shows methods based on:
Use of 3D CAD models and matching of segmented elements;
Extraction of invariant geometric features from the 3D map;
Superquadrics;
Spin images;
Tensors;
3D template matching
In this work 3D template matching: is used:
the point cloud of a segmented object is compared to the point cloud of a template;
ability to treat complex shapes that cannot be modelled by local features

6. Our approach To satisfy industrial needs in terms of:
short time-to-market
High cost effectiveness
Ease of the implementation of the solution
We chose to follow a building brick philosophy based on:
The use of a market available acquisition sensor;
The use of open source code for scene segmentation:
Point Cloud Library (PCL) platform
The use of market available libraries for the object identification problem:
Shape Analysis Library (SAL3D, by AQSense Inc., Spain)

7. 3D data acquisition: the principle
Optical active triangulation using a laser blade and a video camera
The laser blade is deformed by the object shape
The triangulation geometry allows us to correlate the deformation to the object surface
Full 3D recontruction is achived by scanning the whole scene along the direction perpendicular to the laser blade (the robot is used to scan the scene)
A very fast device (Sick Ranger 50D) is used to acquire the laser blades and to provide the (x,z) profiles
Acquired profile
Laser projector z x
Sick ranger 50D y

8. 3D data acquisition: the point cloudsx z y z x y

9. 3D data segmentation 1 2 1 2 3 3 4 4 5 6 5 6 First step:
Outlayer remotion;
Identification and remotion of points belonging to the acquisition plane;
Evaluation of the y coordinate 1 2 1 2 3 3 4 4 x 5 6 z 5 6 y

10. 3D data segmentation 1 2 2 Numerosity of points Removed points
Second step:
Evaluation of the mean distance of each cloud point from its neighbours
Analysis of the distribution of mean distances (m)
Detection of ‘isolated’ points: points with a mean distance greater than a predefined threshold (THR)
TRH is evaluated taking into account for the uncertainty of the measured 3D data
Elimination of ‘isolated’ points
Numerosity of points 1 2 2
Removed points
Mean distance
TRH μ

11. 3D data segmentation
An example

12. 3D data segmentation Third step: from points to clusters
Clusters are formed by points sufficiently near to each other
Sufficiently means ‘belonging to a sphere with a predefined threshold
The search is iterative
Different colours mean different objects (clusters)

13. 3D data segmentation Overlap y z x Bounding box 1 Bounding box 2
Fourth step:
remotion of small clusters
Detection of occluded clusters
A 2D approach is implemented:
Evaluation of the bounding boxes of each cluster and analysis of their dimensions
Overlap
Bounding box 1 y
Bounding box 2 z x

14. Matching
Templates are acquired of the object to be recognized, and saved into a database.
Each cluster is matched to each template.
A disparity map is generated.
The best matching is detected using suitable error and overlapping parameters

15. The software platform
OptoRanger: C++, wxWidgets, Point Cloud Library, SAL3D.
Translation: Definition of the reference system.
Models: Creation of the template database.
Matching: Acquisition, segmentation, detection.

16. Experimental validation
Le prestazioni del sistema sono valutate simulando prove di funzionamento reale, con oggetti di differenti forme e dimensioni disposti casualmente. OK OK OK OK OK OK OK NO
3 cm
4 cm
1,4 cm
2 cm
1,8 cm OK
Per ogni scena viene verificata la correttezza nel riconoscimento del tipo di oggetto e delle coordinate di posizione ed orientamento. OK OK OK OK OK OK

17. Conclusions
The system efficiently detects different objects for bin picking applications.
The advantages are (i) the possibility of recognising objects with arbitrary shape without prior knowledge and (ii) the flexibility added to the process.
Future work
To characterise the system behaviour on metallic, reflective surfaces.
To characterise the system behaviour on real industrial processes, with the integration of a manipulator.