1. SL-2005-009-a Robot Vision 3 exercise
January 31 , 2005

2. Robot Vision 3 exercise

3. Scorpion profiles RobotVision3_Start.zip RobotVision3_Final.zip
No configuration but images
RobotVision3_Final.zip
Final configuration with all tools
Both profiles are on the Scorpion CD in the UnSupportedProfile folder
SL a Robot Vision 3 exercise
January 31 , 2005

4. The Task
To locate the object and find the x and y co-ordinates of the picking point – centre of valve
To measure the rotation (angle)
To send the data (x, y and angle) to the robot
SL a Robot Vision 3 exercise
January 31 , 2005

5. The system Camera: VGA monochrome Communication link: RS 232
Operating procedure:
Robot is the master and Vision system the slave
Robot sends trigger signal
Vision system take an image and do the processing
Vision system sends the result (status, position including angle) to Robot
SL a Robot Vision 3 exercise
January 31 , 2005

6. Vision strategy – part 1
For high accuracy: Use Calibration tool to minimize lens distortion
To reduce varying light conditions: Use ImageConverter tool to minimize reflections
For rough location and rotation: Use TemplateFinder2 tool to locate the object and the rotation
SL a Robot Vision 3 exercise
January 31 , 2005

7. Vision strategy – part 2
Use PolygonMatch tool to fine-tune the picking point and rotation given by the TemplateFinder2 tool
SL a Robot Vision 3 exercise
January 31 , 2005

8. Vision strategy – part 3
Create a reference system based on the result from the PolygonMatch tool, with visualisation in the original image
SL a Robot Vision 3 exercise
January 31 , 2005

9. Complete Tools list SL-2005-009-a Robot Vision 3 exercise
January 31 , 2005

10. Lens calibration
In robot vision application the camera often have a large Field Of View (FOW) and a relative short distance to the object
This require a wide angle lens and consequently large lens distortion
Without correcting the distortion, the inaccuracy in the outer part of the FOW might cause the robot to fail picking the object correctly
SL a Robot Vision 3 exercise
January 31 , 2005

11. 4 point robot calibration
By using 4 point calibration you can establish a co-ordinate system equal to the one used by the robot
By moving the robot to the centre of each point the robot co-ordinates are found
In Scorpion the centre of each is found in the Scorpion reference system
This enable you to communicate pick up points and rotation in the robot co-ordinate system
SL a Robot Vision 3 exercise
January 31 , 2005

12. Understanding lens correction and robot calibration
Correction of lens distortion is independent of distance to the object
The lens correction is done in one plane and valid for all robot planes
Robot calibration is only valid for a specific distance to the object (Robot plane)
For each different robot picking plane a individual robot calibration must be done
Scorpion can handle multiple robot planes
Robot plane 3
Robot plane 2
Robot plane 1
SL a Robot Vision 3 exercise
January 31 , 2005

13. Pixel co-ordinate system
Origo
X=0, Y=0
X=0, Y=640
Default co-ordinate system with Origin in upper left corner
X and Y pixels according to camera resolution
VGA 760 x 480 (used in the exercise)
XGA 1024 x 768 Y X
X=480, Y=640
X=480, Y=0
SL a Robot Vision 3 exercise
January 31 , 2005

14. Configure the Calibration tool
Open the Calibrator tool and name it “GeoCalibrator”
Set number of rows and column – count from image
Set row and column spacing to 10 mm, and the unit to mm (millimetre)
Click on compute 2 1 3
SL a Robot Vision 3 exercise
January 31 , 2005

15. Calibrator – advanced setup
The calibrator use blob technology to find the black spots on the grid
If “Active” is ticked ON in the Advanced tab, user defined configuration of the blob search parameter can be entered.
Only to be used if the default values are not working satisfactory
SL a Robot Vision 3 exercise
January 31 , 2005

16. Calibrator visualisation
Both the actual corrections (Non-linear distortion) and residues (remaining distortion after correction) can be visualised
You can increase the gain factor to improve the visualisation
SL a Robot Vision 3 exercise
January 31 , 2005

17. The calibrator result
When the relevant indicator is green (Gauging or Robot Vision) the accuracy is accepted for that type of application
In the Result of Model fit panel you will find more details:
SL a Robot Vision 3 exercise
January 31 , 2005

18. Multi co-ordinate system
Pixel co-ordinate system (red)
the default coordinate system
Scaling is in pixels
Calibration co-ordinate system (yellow)
Generated by the Calibration tool
Lens distortion is compensated
Scaling is in mm Y X Y X
SL a Robot Vision 3 exercise
January 31 , 2005

19. Get image for Robot calibration
The image used is in the folder calib.
Go to tab “Camera” and double click on “Robot Vision” Image Settings and select the calib folder
Click on Snapshot to get the image
SL a Robot Vision 3 exercise
January 31 , 2005

20. Creating robot co-ordinates – step 1
Use the mouse (right click and select view info) to find the co-ordinates of the four points.
Hint:
Use zoom to get accuracy
Remember to select GeoCalibrator as reference system (right click and select reference system to check)
Point 1
Point 2
Point 3
Point 4
SL a Robot Vision 3 exercise
January 31 , 2005

21. Creating robot co-ordinates – step 2
Open ExternalReference tool and name it “RobotCoordinates”
Enter both local and robot co-ordinates in the External tool
Remember to select the tool named “GeoCalibrator” as reference, since the local co-ordinates is relative to that system
SL a Robot Vision 3 exercise
January 31 , 2005

22. Move reference system to image
The origin of the robot coordinates are far outside the image.
To see the result we can move it inside the image using the tool MoveReference
Name the tool “MoveRobotCoordToImage” and use the inputs as on the image to the right
SL a Robot Vision 3 exercise
January 31 , 2005

23. Robot coordinate system
We now have 3 coordinate systems
1: Pixel
2: GeoCalibrator
3: RobotCoordinates (moved)
Remember to change back to the image folder “images” X Y X Y X
SL a Robot Vision 3 exercise
January 31 , 2005

24. Eliminating reflections
Varying reflections from the valve may cause problems
To eliminate the reflections with can make a new manipulated image.
First add a new image in “Image Setting” under the camera tab and name it “Valve”
Remember to tick OFF the active mark 1 2 3 4
SL a Robot Vision 3 exercise
January 31 , 2005

25. Reducing the contrast
By using the Image Converter we can create a new picture
Select the ImageConverter tool and name it “ReduceReflections”
Activate the Intensity part and write in the name of the image “Valve” created before
To eliminate the reflections set the Filter High = 150 and set = 150
This means that all pixels with greyscale values over 150 are set to 150
SL a Robot Vision 3 exercise
January 31 , 2005

26. How to use a template finder1
Select TemplatFinder2 tool and name it “FindParts”
We will use image 2 (Enter 2 in the Image Index field under the General tab)
To include the lens correction and to make it work in robot co-ordinates use the tool RobotCoordinates as reference
The search area is the complete image, but to secure that parts close to the image border is found, make the search area slightly bigger than the image 2
SL a Robot Vision 3 exercise
January 31 , 2005

27. Set resampling
To speed up the processing time we will use resampling to reduce the information in the image
Set the new image size to 125. This will give approximately 340 x 260 pixels, a reduction of 50 %
SL a Robot Vision 3 exercise
January 31 , 2005

28. Look at resampled image1
Go to Resampling Tab
When you have the image you want to use as the template push “Acquire resampled image”
This picture have 50% less pixels than the original and is corrected for the lens distortion
The more you reduce the number of pixels the faster the template will work but consequently less accurate 2
SL a Robot Vision 3 exercise
January 31 , 2005

29. Create template image Use the cursor to zoom around the part
Try to make a square image. This will work more efficient when rotation is required
Push “Copy selection to clipboard”
SL a Robot Vision 3 exercise
January 31 , 2005

30. Store Templates
Click on the paste knob and the template is store in template “1”
If “Paste square image” is ticked on, pixels are added to make it 100% square
You can add tabs for more images as templates by pushing “+” 1
SL a Robot Vision 3 exercise
January 31 , 2005

31. Visualisation
Push apply and look in image to see if the part is found. If no results adjust the threshold to a lower value
Adjust the visualisation as indicated to the right and in the image you will have the template axis and score
SL a Robot Vision 3 exercise
January 31 , 2005

32. Understanding the results
Go to the result tab and you will find all results
Number of matches is number of parts found
Match center (x,y) for each match plus angle
Score for each match
Template number and Template name (if any)
SL a Robot Vision 3 exercise
January 31 , 2005

33. Set new centre and offset angle
Centre of the template image is the default centre.
You can change the centre by moving the cursor to the point where you want the centre, right click and select “Set center”
Before
After
SL a Robot Vision 3 exercise
January 31 , 2005

34. Adjust offset angle
The angle of the axis system is dependent of the rotation of the template image
You can offset the angle to make the axis match any orientation
You can also give the Template a name (Very useful with many templates)
SL a Robot Vision 3 exercise
January 31 , 2005

35. Advanced Tab: Template decimation
Reduce the pixel content in the image to speed up the tool
Start with Automatic settings. While watching the analysis time, adjust the factor manually 1 or 2 steps up and down to find the “fastest” factor
SL a Robot Vision 3 exercise
January 31 , 2005

36. Advanced Tab: Template modification
Handles rotation and scaling of the template
We will use the template finder to roughly find the rotation (+/- 10 degrees)
Template finder can also handle variation in scaling
Both rotation and scaling takes time and processing capacity
SL a Robot Vision 3 exercise
January 31 , 2005

37. Advanced Tab: Match description
Select what to visualise in the description field
SL a Robot Vision 3 exercise
January 31 , 2005

38. Advanced Tab: Sorting results
Set criteria for sorting the parts found
Organise the matches according to score, x or y position or in rows and coloumns
Useful when you want a robot to pick in a certain order
SL a Robot Vision 3 exercise
January 31 , 2005

39. Valve located
To check that a valve is located we can use the result from the TenplateFinder
Use logic tool and select the parameter “Number of matches”
The minimum condition should 1
SL a Robot Vision 3 exercise
January 31 , 2005

40. Fine-tune the rotation angle and centre point (picking point)
Create a PolygonMatch tool named “FindRotation”
The tool should reference to the TemplateFinder “FindParts” and will then generate a correction angle and centre offset value relative to the results from the TemplateFinder
These values can be used to establish an accurate angle and picking point
SL a Robot Vision 3 exercise
January 31 , 2005

41. How does it work
Diagonal on the polygon, a number of trace lines are set. These trace lines detect points as for Linefinder and RadialArcFinder
In this case we will set up the trace lines to find the mid point of the reflections on the tips.
SL a Robot Vision 3 exercise
January 31 , 2005

42. Using multi polygons We will define 3 polygons
Circle
Tip 1 and 2 will correct the rotation, and circle will correct the centre point
Circle
Tip 2
Tip 1
SL a Robot Vision 3 exercise
January 31 , 2005

43. How to create a polygon Keep the Ctrl button down
Then click with the mouse where you want the polygon to be
Click “Paste new” and the polygon is stored in the tool
The polygon is given a number and you can see the coordinates in “Contents”
click
click
click
click
SL a Robot Vision 3 exercise
January 31 , 2005

44. Create all 3 polygons Paste new for each polygon
SL a Robot Vision 3 exercise
January 31 , 2005

45. Configure the PolygonMatch tool
Set visualisation as in the image to the right
SL a Robot Vision 3 exercise
January 31 , 2005

46. Set the Tracelines
To equally weight the results of each of the 3 polygon tick on “Use trace line count”. Now each polygon will have 21 trace lines/points
To make it work the trace lines must be longer. Set start point to – 20 and end point to 20
SL a Robot Vision 3 exercise
January 31 , 2005

47. Set the Edge detection
To find the mid point of the reflections select Differentiation 2.
The smooth base should be 10, equal to the average width of the reflections in pixels
Polarity “Light to dark” since we are looking for light points
And threshold must be reduced to 0.5
SL a Robot Vision 3 exercise
January 31 , 2005

48. Polygon match result
Activate the Child reference base and we have a corrected rotation system
The axes may need an offset to fit as on the image
Before
After
SL a Robot Vision 3 exercise
January 31 , 2005

49. The new rotation system co-ordinate system
The rotation system in PolygonMatch is related to the centre point of the TemplateFinder “FindParts”
It needs to be converted to Robot co-ordinates
SL a Robot Vision 3 exercise
January 31 , 2005

50. Change to Robot co-ordinates
Select the MoveReference Tool and name it “ValveRefSystem”
Reference is made to FindRotation and this tool operates as the FindRotation reference system
In the result tab select RobotCoordinates as result reference
SL a Robot Vision 3 exercise
January 31 , 2005

51. Visualisation of rotation
Now the Rotation system is in Robot co-ordinates
In tab “General” you can select ImageIndex1 to move the visualisation to the original image
SL a Robot Vision 3 exercise
January 31 , 2005

52. Customise text and visualisation
We will use Python scripting to make a nice text string to present the result for the operator
We will make a simple visualisation of the picking point in the original image
SL a Robot Vision 3 exercise
January 31 , 2005

53. Python scripting Open a Python tool and name it “CreatePositionString”
The script on next page collect the x.y co-ordinates and angle from the Valve Reference System and create a string that is store in the text result field of the tool
And create a circle marker at the picking point
SL a Robot Vision 3 exercise
January 31 , 2005

54. Python script valvelocated = GetValue('ValveLocated.Value')
if valvelocated:
x = GetValue('ValveRefSystem.Origo_x')
y = GetValue('ValveRefSystem.Origo_y')
a = GetValue('ValveRefSystem.Rotation')
str0 = 'Position = (%(x).1f , %(y).1f) [mm] Angle = %(a).1f [degrees]' %vars()
SetValue('CreatePositionString.Text',str0)
#print str0
#This line make maker at the picking point
DrawMarker('RobotCoordinates', x, y, 'blue', 8 , 1)
else:
SetValue('CreatePositionString.Text','No position')
#Hint! Copy this page directly into the Python tool
SL a Robot Vision 3 exercise
January 31 , 2005

55. Adding text string to the result panel
Right click on the Result Panel and set Number of measured values to 1
Right click again and for measured values select the right parameter
Tool: CreatePositionString
Parameter: Text
SL a Robot Vision 3 exercise
January 31 , 2005

56. Operator interface SL-2005-009-a Robot Vision 3 exercise
January 31 , 2005

57. Sending data to the robot
Each time a valve is located the x, y and angle values should be sent over RS232 to the robot
In the event that a valve is NOT located a the figure 0 should be sent out
These commands can be configured in the “Valve located” and “Valve not found” states defined under the “Settings” mode
These commands are only executed when the state is true
SL a Robot Vision 3 exercise
January 31 , 2005

58. Configure Valve located command1
Go to the “Settings” mode and open “Valve located” state
Select the Command tab and click on new
Click on the … button to right of the Command line and select RS232Cmd from the list 2 4 3 5 6
SL a Robot Vision 3 exercise
January 31 , 2005

59. Writing the parameters
To send the x values use the following syntax:
%0.1fValveRefSystem.Origo_x
% = configuration of a figure
0.1f=floating point with 1 decimal configuration
ValveRefSystem.Origo_x = Parameter to be sent
SL a Robot Vision 3 exercise
January 31 , 2005

60. The complete parameter string is:
Add then the y value and angle
Each figure is separated by a comma
%0.1fValveRefSystem.Origo_x;%0.1fValveRefSystem.Origo_y;%.0fValveRefSystem.Rotation
SL a Robot Vision 3 exercise
January 31 , 2005

61. Valve not located command
Add a RS232 command in the Valve not found State with the Parameter 0.
SL a Robot Vision 3 exercise
January 31 , 2005

62. Checking communication
In the communication Tab both out and in going communication can be monitored
Select RS232 tab and check the results
SL a Robot Vision 3 exercise
January 31 , 2005

63. Congratulation
You have made an advanced Robot Vision system using Template Finder 2 and Polygon Match
SL a Robot Vision 3 exercise
January 31 , 2005

64. SL-2005-009-a Robot Vision 3 exercise
January 31 , 2005