Robot guidance using Matrox Design Assistant vision software

Robot guidance using Matrox Design Assistant February 2015 About the Matrox group » Established in 1976 » Privately held » 800 employees worldwide » Headquartered in Montréal, Canada » Three (3) divisions » Graphics: solutions for multi-display systems » Video: solutions for video production » Imaging: solutions for machine vision

Robot guidance using Matrox Design Assistant February 2015 About Matrox Imaging » Leading developer of component-level hardware and software solutions » Essentially based in Montréal, Canada » 150 employees of which 60% in R&D » Worldwide presence through third-parties » Supplier to OEMs and integrators

Robot guidance using Matrox Design Assistant February 2015 Matrox Imaging product line Vision software Matrox Imaging Library (MIL) and Matrox Design Assistant Smart cameras Vision systems / industrial computers Frame grabbers Analog, Camera Link®, Camera Link HS™, CoaXpress®, DVI-D and SDI Vision processors Frame grabbers with FPGA-based processing

Robot guidance using Matrox Design Assistant February 2015 Matrox Design Assistant » Integrated Development Environment (IDE) for designing vision applications » Construct and configure logic using a flowchart » Layout and configure operator interface (HMI) » Deploy project to Matrox smart camera, vision system or any PC

Robot guidance using Matrox Design Assistant February 2015 Ver 1.0 (2005) Ver 2.0 (2008) Ver 3.0 (2012) Ver 4.0 (2014/5) Ver SP1 (2015) im.canon-its.jp Matrox Design Assistant » DA 1.0 – DA 3.0 » Matrox IRIS GT » DA 4.0 » Matrox IRIS GT » Matrox 4Sight-GPm » General PC » GigE Vision » DA 4.0 SP1 » Denso Robot » USB3 Vision

Robot guidance using Matrox Design Assistant February 2015 Matrox Design Assistant IDE Contextual guide for flowchart step View flowchart with image or operator interface layout Navigate flowchart View flowchart results Test and run flowchart steps Configure flowchart step

Robot guidance using Matrox Design Assistant February 2015 Flowchart steps » Flow control » Image processing and analysis » Enhance and transform images » Locate objects » Extract and measure features » Read character strings » Decode identification marks » I/O and communication

Robot guidance using Matrox Design Assistant February 2015 System integration » Matrox Design Assistant projects interface to automation equipment and enterprise systems through » Ethernet protocols (EtherNet/IP™, MODBUS®, PROFINET and TCP/IP) » native robot interfaces (ex. DENSO) » discrete digital I/Os (on Matrox smart camera and vision system)

Robot guidance using Matrox Design Assistant February 2015 Run-time operator interface Image with annotations Accessed from Internet Explorer® Flimstrip view Operator inputs Inspection results Inspection status

Robot guidance using Matrox Design Assistant February 2015 Deployment options

Robot guidance using Matrox Design Assistant February 2015 Matrox Iris GT smart camera » Efficient Intel® Atom® processor » Choice of different sensor resolutions and speed » Gigabit Ethernet, KVM and GPIO interfaces

Robot guidance using Matrox Design Assistant February 2015 Matrox 4Sight GPm vision system » Fanless design in a small rugged housing » Capable 3rd generation Intel® Core® processor » Multiple PoE and USB 3.0 ports for GigE Vision® and USB3 Vision™ cameras » Real-time digital I/Os

Robot guidance using Matrox Design Assistant February 2015 Robot guidance with DENSO-Matrox » 2D vison-guided robotics » Objects to locate and pick rest on a common plane » Fixed camera viewpoint » Camera mounted overhead or on end of robot arm

Robot guidance using Matrox Design Assistant February 2015 Prerequisites DENSO » RobCom COA provider for RC8 robot controller Matrox » Matrox Design Assistant 4 with Service Pack 1

Robot guidance using Matrox Design Assistant February SightGPm Gigabit Ethernet Matrox IRIS GT 1200 Matrox4SightGPm DA4.0 IDE DENSO VS-050 Switching HUB Prerequisites Development Runtime

Robot guidance using Matrox Design Assistant February 2015 Network socket-based communication Initialize Listen Connect Send request Send results Receive request Verify connection Receive results Socket clientSocket server

Robot guidance using Matrox Design Assistant February 2015 RC8 programming (PacScript) » Bind socket » Wait for connection » Send request to locate object » Obtain location of object » Etc. … SendAndRecieve: Do ' Move to home position Move HOME_POSE ' Open the gripper MoveGripper &hA0, &hFF, &hFF ' Send current position and request the sensor to find the object Call Send(0, caoCtrl) ' Get the position of the object P20 = Receive(caoCtrl) If PosX(P20) == 0 AND PosY(P20) == 0 AND PosZ(P20) == 0 then Goto SendAndRecieve End If ' Move over the object Dim OverPoint as Position OverPoint = P20 LetZ OverPoint = PosZ(OverPoint) + 50 Move OverPoint delay 500 ' Move to pick the object Move P20 delay 500 ' Close the gripper to pick the object MoveGripper &hE0, &hFF, &hFF …

Robot guidance using Matrox Design Assistant February 2015 Matrox Design Assistant project » Configure platform to connect to RC8 controller

Robot guidance using Matrox Design Assistant February 2015 Matrox Design Assistant project » Add flowchart steps » RobotWait » Wait for request from robot program to locate next object » Request includes current robot pose sextet: X, Y, Z, R X, R Y, R Z » RobotWriter » Return object location (pose) to robot program » Returns pose sextet: X, Y, Z, R X, R Y, R Z (Z, R X, R Y unused)

Robot guidance using Matrox Design Assistant February 2015 Calibration considerations » Establish common coordinate system for robot and vision system » Robot and vision system are calibrated separately using same reference points

Robot guidance using Matrox Design Assistant February 2015 Calibration considerations » Robot is calibrated by » moving its tool to each reference point » recording each position in the robot’s coordinate system

Robot guidance using Matrox Design Assistant February 2015 Calibration considerations » Vision system is calibrated by associating each robot position with the same point in the image Vision system is then able to return positional results in the robot’s coordinate system

Robot guidance using Matrox Design Assistant February 2015 Calibration considerations » Can do quasi 3D robot guidance by » calibrating at multiple known planes » switching calibrations when switching work planes Image from websitewww.hamiltonrobotics.com

Robot guidance using Matrox Design Assistant February 2015 Matrox Design Assistant project » Choice of locator steps » BlobAnalysis step » PatternMatching step » ModelFinder step

Robot guidance using Matrox Design Assistant February 2015 BlobAnalysis step » Locate objects by finding regions of connected pixels (blobs) » Must be able to separate foreground (object) from background pixels » Special pre-processing may be needed » Additional math may be needed to return X, Y, R Z » Fastest method

Robot guidance using Matrox Design Assistant February 2015 PatternMatching step » Locate objects by looking for a similar distribution of intensity » Directly returns X, Y » Can also return R Z but not the most efficient method » Not suited if subject to non-uniform changes in illumination » Fast method

Robot guidance using Matrox Design Assistant February 2015 ModelFinder step » Locate objects by looking for geometric features (edges) » Directly returns X, Y, R Z » Handles occlusion and non-uniform changes in illumination » Not the fastest method

Robot guidance using Matrox Design Assistant February 2015 Application with Matrox GPm H/W TriggerStrobe Light Operator View

Robot guidance using Matrox Design Assistant February 2015 请到北京三宝的展台（ N1 #1242 ）观看实体演示