SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Robot Vision 3 exercise

Scorpion profiles RobotVision3_Start.zip RobotVision3_Final.zip No configuration but images RobotVision3_Final.zip Final configuration with all tools Both profiles are on the Scorpion CD in the UnSupportedProfile folder SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

The Task To locate the object and find the x and y co-ordinates of the picking point – centre of valve To measure the rotation (angle) To send the data (x, y and angle) to the robot SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

The system Camera: VGA monochrome Communication link: RS 232 Operating procedure: Robot is the master and Vision system the slave Robot sends trigger signal Vision system take an image and do the processing Vision system sends the result (status, position including angle) to Robot SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Vision strategy – part 1 For high accuracy: Use Calibration tool to minimize lens distortion To reduce varying light conditions: Use ImageConverter tool to minimize reflections For rough location and rotation: Use TemplateFinder2 tool to locate the object and the rotation SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Vision strategy – part 2 Use PolygonMatch tool to fine-tune the picking point and rotation given by the TemplateFinder2 tool SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Vision strategy – part 3 Create a reference system based on the result from the PolygonMatch tool, with visualisation in the original image SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Complete Tools list SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Lens calibration In robot vision application the camera often have a large Field Of View (FOW) and a relative short distance to the object This require a wide angle lens and consequently large lens distortion Without correcting the distortion, the inaccuracy in the outer part of the FOW might cause the robot to fail picking the object correctly SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

4 point robot calibration By using 4 point calibration you can establish a co-ordinate system equal to the one used by the robot By moving the robot to the centre of each point the robot co-ordinates are found In Scorpion the centre of each is found in the Scorpion reference system This enable you to communicate pick up points and rotation in the robot co-ordinate system SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Understanding lens correction and robot calibration Correction of lens distortion is independent of distance to the object The lens correction is done in one plane and valid for all robot planes Robot calibration is only valid for a specific distance to the object (Robot plane) For each different robot picking plane a individual robot calibration must be done Scorpion can handle multiple robot planes Robot plane 3 Robot plane 2 Robot plane 1 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Pixel co-ordinate system Origo X=0, Y=0 X=0, Y=640 Default co-ordinate system with Origin in upper left corner X and Y pixels according to camera resolution VGA 760 x 480 (used in the exercise) XGA 1024 x 768 Y X X=480, Y=640 X=480, Y=0 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Configure the Calibration tool Open the Calibrator tool and name it “GeoCalibrator” Set number of rows and column – count from image Set row and column spacing to 10 mm, and the unit to mm (millimetre) Click on compute 2 1 3 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Calibrator – advanced setup The calibrator use blob technology to find the black spots on the grid If “Active” is ticked ON in the Advanced tab, user defined configuration of the blob search parameter can be entered. Only to be used if the default values are not working satisfactory SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Calibrator visualisation Both the actual corrections (Non-linear distortion) and residues (remaining distortion after correction) can be visualised You can increase the gain factor to improve the visualisation SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

The calibrator result When the relevant indicator is green (Gauging or Robot Vision) the accuracy is accepted for that type of application In the Result of Model fit panel you will find more details: SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Multi co-ordinate system Pixel co-ordinate system (red) the default coordinate system Scaling is in pixels Calibration co-ordinate system (yellow) Generated by the Calibration tool Lens distortion is compensated Scaling is in mm Y X Y X SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Get image for Robot calibration The image used is in the folder calib. Go to tab “Camera” and double click on “Robot Vision” Image Settings and select the calib folder Click on Snapshot to get the image SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Creating robot co-ordinates – step 1 Use the mouse (right click and select view info) to find the co-ordinates of the four points. Hint: Use zoom to get accuracy Remember to select GeoCalibrator as reference system (right click and select reference system to check) Point 1 Point 2 Point 3 Point 4 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Creating robot co-ordinates – step 2 Open ExternalReference tool and name it “RobotCoordinates” Enter both local and robot co-ordinates in the External tool Remember to select the tool named “GeoCalibrator” as reference, since the local co-ordinates is relative to that system SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Move reference system to image The origin of the robot coordinates are far outside the image. To see the result we can move it inside the image using the tool MoveReference Name the tool “MoveRobotCoordToImage” and use the inputs as on the image to the right SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Robot coordinate system We now have 3 coordinate systems 1: Pixel 2: GeoCalibrator 3: RobotCoordinates (moved) Remember to change back to the image folder “images” X Y X Y X SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Eliminating reflections Varying reflections from the valve may cause problems To eliminate the reflections with can make a new manipulated image. First add a new image in “Image Setting” under the camera tab and name it “Valve” Remember to tick OFF the active mark 1 2 3 4 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Reducing the contrast By using the Image Converter we can create a new picture Select the ImageConverter tool and name it “ReduceReflections” Activate the Intensity part and write in the name of the image “Valve” created before To eliminate the reflections set the Filter High = 150 and set = 150 This means that all pixels with greyscale values over 150 are set to 150 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

How to use a template finder 1 Select TemplatFinder2 tool and name it “FindParts” We will use image 2 (Enter 2 in the Image Index field under the General tab) To include the lens correction and to make it work in robot co-ordinates use the tool RobotCoordinates as reference The search area is the complete image, but to secure that parts close to the image border is found, make the search area slightly bigger than the image 2 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Set resampling To speed up the processing time we will use resampling to reduce the information in the image Set the new image size to 125. This will give approximately 340 x 260 pixels, a reduction of 50 % SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Look at resampled image 1 Go to Resampling Tab When you have the image you want to use as the template push “Acquire resampled image” This picture have 50% less pixels than the original and is corrected for the lens distortion The more you reduce the number of pixels the faster the template will work but consequently less accurate 2 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Create template image Use the cursor to zoom around the part Try to make a square image. This will work more efficient when rotation is required Push “Copy selection to clipboard” SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Store Templates Click on the paste knob and the template is store in template “1” If “Paste square image” is ticked on, pixels are added to make it 100% square You can add tabs for more images as templates by pushing “+” 1 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Visualisation Push apply and look in image to see if the part is found. If no results adjust the threshold to a lower value Adjust the visualisation as indicated to the right and in the image you will have the template axis and score SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Understanding the results Go to the result tab and you will find all results Number of matches is number of parts found Match center (x,y) for each match plus angle Score for each match Template number and Template name (if any) SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Set new centre and offset angle Centre of the template image is the default centre. You can change the centre by moving the cursor to the point where you want the centre, right click and select “Set center” Before After SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Adjust offset angle The angle of the axis system is dependent of the rotation of the template image You can offset the angle to make the axis match any orientation You can also give the Template a name (Very useful with many templates) SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Advanced Tab: Template decimation Reduce the pixel content in the image to speed up the tool Start with Automatic settings. While watching the analysis time, adjust the factor manually 1 or 2 steps up and down to find the “fastest” factor SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Advanced Tab: Template modification Handles rotation and scaling of the template We will use the template finder to roughly find the rotation (+/- 10 degrees) Template finder can also handle variation in scaling Both rotation and scaling takes time and processing capacity SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Advanced Tab: Match description Select what to visualise in the description field SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Advanced Tab: Sorting results Set criteria for sorting the parts found Organise the matches according to score, x or y position or in rows and coloumns Useful when you want a robot to pick in a certain order SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Valve located To check that a valve is located we can use the result from the TenplateFinder Use logic tool and select the parameter “Number of matches” The minimum condition should 1 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Fine-tune the rotation angle and centre point (picking point) Create a PolygonMatch tool named “FindRotation” The tool should reference to the TemplateFinder “FindParts” and will then generate a correction angle and centre offset value relative to the results from the TemplateFinder These values can be used to establish an accurate angle and picking point SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

How does it work Diagonal on the polygon, a number of trace lines are set. These trace lines detect points as for Linefinder and RadialArcFinder In this case we will set up the trace lines to find the mid point of the reflections on the tips. SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Using multi polygons We will define 3 polygons Circle Tip 1 and 2 will correct the rotation, and circle will correct the centre point Circle Tip 2 Tip 1 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

How to create a polygon Keep the Ctrl button down Then click with the mouse where you want the polygon to be Click “Paste new” and the polygon is stored in the tool The polygon is given a number and you can see the coordinates in “Contents” click click click click SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Create all 3 polygons Paste new for each polygon SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Configure the PolygonMatch tool Set visualisation as in the image to the right SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Set the Tracelines To equally weight the results of each of the 3 polygon tick on “Use trace line count”. Now each polygon will have 21 trace lines/points To make it work the trace lines must be longer. Set start point to – 20 and end point to 20 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Set the Edge detection To find the mid point of the reflections select Differentiation 2. The smooth base should be 10, equal to the average width of the reflections in pixels Polarity “Light to dark” since we are looking for light points And threshold must be reduced to 0.5 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Polygon match result Activate the Child reference base and we have a corrected rotation system The axes may need an offset to fit as on the image Before After SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

The new rotation system co-ordinate system The rotation system in PolygonMatch is related to the centre point of the TemplateFinder “FindParts” It needs to be converted to Robot co-ordinates SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Change to Robot co-ordinates Select the MoveReference Tool and name it “ValveRefSystem” Reference is made to FindRotation and this tool operates as the FindRotation reference system In the result tab select RobotCoordinates as result reference SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Visualisation of rotation Now the Rotation system is in Robot co-ordinates In tab “General” you can select ImageIndex1 to move the visualisation to the original image SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Customise text and visualisation We will use Python scripting to make a nice text string to present the result for the operator We will make a simple visualisation of the picking point in the original image SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Python scripting Open a Python tool and name it “CreatePositionString” The script on next page collect the x.y co-ordinates and angle from the Valve Reference System and create a string that is store in the text result field of the tool And create a circle marker at the picking point SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Python script valvelocated = GetValue('ValveLocated.Value') if valvelocated: x = GetValue('ValveRefSystem.Origo_x') y = GetValue('ValveRefSystem.Origo_y') a = GetValue('ValveRefSystem.Rotation') str0 = 'Position = (%(x).1f , %(y).1f) [mm] Angle = %(a).1f [degrees]' %vars() SetValue('CreatePositionString.Text',str0) #print str0 #This line make maker at the picking point DrawMarker('RobotCoordinates', x, y, 'blue', 8 , 1) else: SetValue('CreatePositionString.Text','No position') #Hint! Copy this page directly into the Python tool SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Adding text string to the result panel Right click on the Result Panel and set Number of measured values to 1 Right click again and for measured values select the right parameter Tool: CreatePositionString Parameter: Text SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Operator interface SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Sending data to the robot Each time a valve is located the x, y and angle values should be sent over RS232 to the robot In the event that a valve is NOT located a the figure 0 should be sent out These commands can be configured in the “Valve located” and “Valve not found” states defined under the “Settings” mode These commands are only executed when the state is true SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Configure Valve located command 1 Go to the “Settings” mode and open “Valve located” state Select the Command tab and click on new Click on the … button to right of the Command line and select RS232Cmd from the list 2 4 3 5 6 SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Writing the parameters To send the x values use the following syntax: %0.1fValveRefSystem.Origo_x % = configuration of a figure 0.1f=floating point with 1 decimal configuration ValveRefSystem.Origo_x = Parameter to be sent SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

The complete parameter string is: Add then the y value and angle Each figure is separated by a comma %0.1fValveRefSystem.Origo_x;%0.1fValveRefSystem.Origo_y;%.0fValveRefSystem.Rotation SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Valve not located command Add a RS232 command in the Valve not found State with the Parameter 0. SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Checking communication In the communication Tab both out and in going communication can be monitored Select RS232 tab and check the results SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

Congratulation You have made an advanced Robot Vision system using Template Finder 2 and Polygon Match SL-2005-009-a Robot Vision 3 exercise January 31 , 2005

SL-2005-009-a Robot Vision 3 exercise January 31 , 2005