IPD Technical Conference February 19 th 2008 New Processors in Sherlock 7 Ben Dawson

Sherlock’s Processors  Preprocessor = Image to Image (e.g. threshold)  Algorithm = Image to “readings” (e.g. blob analysis)  Formula = Reading to Reading (e.g. add to a number)  Sherlock 7 inherited most Sherlock 6 Processors  Some have slight differences (e.g. dynamic threshold)

New Processors  Some processors improved (e.g., edge detectors)  New processors for new image types (e.g., color)  New processors for specific tasks (e.g., a “bead tool”)  New “high level” processors (e.g., Hough transform)  New utility processors (e.g., test patterns)  Introduce some of these new goodies and application

Rewrite of Edge Detection Algorithms  Our edge detection algorithms needed improvement:  Inconsistent implementations with varying accuracy  Limited options  Sometimes not intuitive to use  Rewrote most standard edge detectors  Improved and consistent implementation  Better accuracy (1/8 pixel nominal, 1/25 best)  Flexible and easy-to-use GUI

GUI for New Edge Detectors

Comparing Old and New Edge Detection  Old edge detectors listed at the bottom of the line algorithms and marked “(legacy)”  Will be deprecated  NOTE: 0,0 is the CENTER of the pixel “Legacy”New, with GUI Detect Edges First EdgeFind Edge Inside Caliper Max Edge Outside Caliper

Other Edge Detectors  Edge Count uses the old interface and algorithms  HVLine has poor sub-pixel accuracy (½ pixel at best)  New edge detectors for specific tasks:  Laser Caliper (also used on Bead Tool)  Corner Detector  Ramp Edges has been subsumed by Detect Edges, etc.  Chatter Edges is an edge enhancer not a edge detector!

Color Processing  Not calibrated (referenced to some standard)  Need standard lighting and calibration targets  Newer DALSA cameras will have calibration  Usually not necessary in machine vision  Can compensate for lighting changes  Color Correction Coefs and Color Correction  Needs a “reference patch” in field-of-view  Even LEDs change color with temperature and age

Some Color Preprocessors  Color Correction – Applies correction coefficients  Gamma – Applies gamma correction  Raises each pixel to a fixed exponent, p g  Makes the image look better on the display  Usually not good for MV. Turn it off at the camera too!  Threshold – AND or OR of R,G,B thresholds  Threshold Components – Threshold individual components  Simple “classifier” that divides color space into cubes  Normalize by Chroma – Divides out intensity

Tray of Aerators

Threshold Components

Normalize by Chroma

Color Algorithms – Statistics  Color Correction Coefs – Learns correction coefficients  Average [channel] – Average value per channel  Count [channel] – Per channel count of pixels with specified value  Count [color] – Count of pixels with specified color  MinMax – Minimum and maximum RGB and location  MinMax [channel] – Minimum and maximum value per channel and location  Statistics [channel] – Arrays of minimum, maximum, average, variance per channel, and histograms  Unique Colors – Number of unique colors in ROI

Color Classifiers  “Recognizes” or “Identifies” learned colors  GUI for training makes our classifiers easy to use  Color Map – Labels learned colors (outputs image)  Color Presence – Lists learned colors found in ROI  Spot Meter – Detects average learned color in an ROI  Trained classifiers can be shared between Color Map and Color Presence  Training for Map and Presence can take some time

Specific Task Processors – Bead Tool  Designed to follow a “bead” – a thin line of material such as glue  Example: Checking glue bead on automotive liners  Set “start box” and learn the path of the bead.  At run time, follows learned path and checks that bead is there and correctly dimensioned

Specific Processors – Chatter Edges  Amplifies very wide (slow intensity change) edges  Designed to help detect bearing “chatter”  Can be used for other slowly changing “ramp” edges  Note the “phase shift” of edges – this is normal

Specific Processors – Laser Tools  Set of tools for mostly doing height measurements using a line of light (like a laser) and triangulation  Laser Caliper – Measure width of a bright line  Similar to Outside caliper but only bright lines and some additional noise reduction  Laser Points – Find line of light points  Laser Line – Fits a Sherlock line to points in the line of light  Laser Height – Measures part heights by triangulation

Laser Tools Setup for Height 1

Laser Tools Setup for Height 2  Can put Laser above or off to the side. Above is better.  Camera must be to the side or above, opposite of laser  DALSA IPD’s height algorithm needs only three height calibration points: baseline, medium, high  NO measurements of the camera and laser positions, angles, distances etc. are needed  Typical accuracy is 1 part in 300  Some limiting factors:  Laser speckle  Lens distortion

High Level Processors  Extract features and information with more constraints and knowledge than edge detectors, calipers, blob, etc.  Roughness – Local standard deviation preprocessor  Texture – Edge Angles – Our first texture analyzer  Edge Crawler Sub-pixel edge crawler (Crawler is pixel)  Corner Finder – Finds corners (duh!)  Hough Transforms – Finds lines, line segments, or circles in noisy images

Roughness Preprocessor  Computes the standard deviation in each neighborhood  Can be used as an “amplifier” for edges  Can be used as a spatial frequency texture filter  Can be used to suppress “background” texture

Roughness as a Texture Filter

Roughness used to Suppress Texture

Texture – Edge Angles  First texture Algorithm (analyzer) – there will be more  Measures edge angle distribution (histogram) and computes an entropy (disorder) measure  These can be used to discriminate different textures

Edge Crawler (sub-pixel)  Tracks edges and reports their sub-pixel position  Can select individual contours  Older Crawler algorithm is integer pixel position

Corner Finder  Finds corners using the Harris corner detector  Corners are more constrained and therefore have more information than edges.

Corner Finder Applications  Applied to finding and counting flexible circuit connector “pins”

Hough Lines  Finds lines in noisy images  Hough transforms are “evidence-based” voting methods

Hough Segments  Finds line segments with specific length ranges  Very useful and works well

Hough Circles  Finds circles with specified radius ranges  Can’t tolerate distortions  Currently difficult to use – often generates a huge number of unwanted circles  Suggest using the spoke tool and BestFitCircleToPts formula for now

Utility Processors – Test Patterns  Test pattern generators  Constant – ROI set to constant color or intensity  Draw Bars – Sub-pixel bars for testing edge detectors  Draw Gaussian – Draws Gaussian intensity distribution  Draw Line – Draws a single line  Draw Ramp – Draws intensity ramps  Draw Grid – Draws a grid of lines of any thicknesses  Draw Checkerboard – Draws checkerboard

Draw Gaussian Example  Many test generators have “blending” option

Checkerboard Example

Testing Connectivity Analysis

Other Utility Processors  Apodize – Increases or decreases intensity towards the edges of the image.  Could be used to compensate for some vignetting  Better to use radial cos 4 (r), not separable functions  ROI to Array(s) – Copies ROI pixel values to array(s)  Border – Puts a border (frame) just inside the ROI  Field Extract – Extracts even or odd fields from monochrome or color images  Mainly used to remove motion “interlace fingers” from older RS-170 interlaced images.  Sometimes useful in surveillance applications

What’s Cooking in the Lab  Adding 16-bit image processors  Only a Statistics algorithm is currently distributed  Averaging, Shading correction  “Smart” conversion to 8-bit images  16-bit test pattern generators  Applications in biological and microscope images  More specific and higher-level processors  Spring Tool (not the season or a delivery time)  Additional Laser Tools (wave, topographic surface, etc.)  Image Morphology Tools (Top-hat, watershed, etc.)  Improvements to Hough and other tools

Summary  Many new and improved processors in Sherlock 7  Most new processors are documented in technical “white papers” found on the web site  Move towards “higher level” vision processors  Edge detection still fundamental, but we can do better in many cases  Ease-of-use is an important design consideration  We welcome your input and suggestions  Send us your hard problems. After we all have a laugh…