1 NIWeek Vision Summit August 2-3, 2011 Austin, Texas

2 Breaking New Ground with Vision Inspection Systems Dr. Dan Milkie Senior Developer Coleman Technologies, Inc. NIWeek Vision Summit August 2–3, 2011 Austin, Texas

What if theres no existing solution? Prototyping tips Planning guide Implementation Imaging algorithms – Finding small defects – 3D laser triangulation Final lessons 3

About us 16 year National Instruments Alliance Partner Basler Vision System Integrator NI Certified Developers & NI Professional Instructors Advanced engineering & science degrees 4

Vision Applications 5 Industrial Research – Dinnerware defect detection system – Dinnerware color pattern inspection system – Robotic seed germination classifier – High-speed seed counting system – Laser drilling inspection – Thin film defect identification – Mirror defect detection – PCB contact pad inspection system – Color tablet tracking – Particle size analysis (powders) – Particle size analysis (liquid suspension) – Glass rod inspection – Well plate inspection system – Bio-sample thermal imaging – Wellplate imaging system – Crystal finder/classifier – Biaxial tissue tester – Two-photon microscopy Plate Inspection System

The Challenge Dinnerware Inspection Many different dishes 1 dish per second Most difficult defect: – White bumps on white plates 6 Defects

Can we image the defects? Prototype with what you have (or loaners) – Area-scan camera (GigE) – Lighting Directional : Desk lamp Broad sources : Room lights, diffusers – NI Vision Development Module Optimize lighting & camera placement – Tip : Replace camera with your eye 7

Okay, I see the defects, but can the computer? 8 Start with NI Vision Assistant – Fastest way to test processing functions – Estimate time using the Performance Meter – Quickly turn scripts into LabVIEW VIs

Okay, I see the defects, but can the computer? Start with NI Vision Assistant – Fastest way to test processing functions – Estimate time using the Performance Meter – Quickly turn scripts into LabVIEW VIs 9

Proof of Principle Are you confident in your plan yet? If not, prototype in LabVIEW! – Combine acquisition and analysis – Tip : Use tester with fresh data or saved image sets Practice good coding style – Prototypes Final version – Good code encourages trying new ideas – Documentation should be automatic 10

Going for it Ready to build your system? Know your test set, criteria Balance goals with sliding scales instead of all-or-nothing Solve most pressing issues first – Example : Our 1 st generation machine tested 1 plate type (their most popular) and only 4 defect types 11 Time $$ Accuracy

Modular, Modular, Modular Independent Stations 1.Bottom view 2.3D imaging 3.Top view Lesson learned : Needed to add baffles Found interference between stations Changed how plates cross gaps Vibration issues 12 NI PCIe-1430 Dual CameraLink NI PCIe-1430 Dual CameraLink NI PCIe x GigE NI PCIe x GigE PC 123

Small defect imaging First version : Next version: Defects show up as dark spots: 13 LED light LED light Line scan Camera Glancing Angle Transmission

Small Defects are Hard to Find! Problems : – Defects are small, low contrast – Large gradients in image – Plate-to-plate variations – Image size (8MP!) – Must process in < 0.5 second Solution: – A custom pixel-by-pixel threshold for each image. 14 Defect!

Step 1 : Unwrap Image Problem : We just need rim pixels Use image masks? – Longer process times – Still have large images Solution : IMAQ Unwrap 8M pixel square -> 1.5M pixel strip Aligns gradients, rim transitions in one direction 15

Step 2 : Create Custom Golden Master What should this image look like if it were perfect? Remove defects using median smoothing – Tip : Use X Size >> Y Size to preserve gradients & edges – Performance Boost : Reduce image resolution, Smooth, then Resample back to original size. 16 Unwrapped Original Median Smoothed Golden Master Defect Defect Removed!

Step 3 : Pixel-by-pixel Threshold 17 Golden Master Results Subtract a constant to set a lower threshold. Original Defect Found! Threshold original image using IMAQ Compare

Benefits of Pixel-by-pixel Thresholds Works with: – Gradients, edges, speckle, large dynamic contrasts Every image checks against itself – Robust against image-image variations, changing lighting conditions Fast (<100ms for 1.5MP) – Limited by smoothing performance 18

3D Imaging Simple inspection for geometric errors – Gouges, bulges, warp 19 Warp examples

Laser Triangulation 20 Laser Line Laser Line Area Camera Area Camera Original Threshold applied

Convert pixels to height 21 With a little more geometry, we can also correct for perspective : WD Laser Line Laser Line Area Camera Area Camera H dy Height =

3D Laser Height Measurement 22 Each camera frame = 1 cross section (per laser line) – Tip : Add multiple laser lines for more cross sections 250 um cross section resolution (100 1 per sec) – Tip : Reduce ROI for fastest frame rates 100 um height resolution

Completed system Fast development with NI tools: – Completed in < 3 months Reliable – Inspected over 1 million dishes Multiple follow-up systems 23

Conclusions Invest in good prototyping practices Prepare for unknown hurdles – Use modular, flexible architectures in hardware, layout, software Defect finding algorithm – Pixel-by-pixel thresholds Simple 3D laser scanner – Inexpensive, easy to setup 24

Questions? 25

26 NIWeek Vision Summit August 2–3, 2011 Austin, Texas Keynotes Industry Keynote: How NI Technology Powers the Space ElevatorLaserMotive Tuesday, August 2 1:00-2:00 Academic Keynote: Industry Trends and Intelligent Production Systems of the Future Interdisciplinary Imaging and Vision Institute, RWTH Aachen University Wednesday, August 3 1:00-2:00 Tuesday, August 2 3D Vision and the KinectNational Instruments10:30-11:30 Wacky Optical Tricks for Machine VisionGE Global Research Center2:15-3:15 Panel Discussion: Latest in Camera Technologies Cyth Systems Basler Vision Technologies Pleora Technologies Toshiba TELI 3:30-4:30 Machine Vision and Industrial Robotics: From Design Concepts to Factory Floor DeploymentImagingLab4:45-5:45 Wednesday, August 3 Precision MetrologyNational Instruments10:30-11:30 Is LabVIEW FPGA Right for My Vision Application?National Instruments2:15-2:45 Autofocus System for an EllipsometerNanometrics Inc.2:45-3:15 Developing a Quality Inspection Method for Selective Laser Melting of Metals Using a High- Speed NIR Camera Katholieke Universiteit Leuven3:30-4:00 Web Inspection of Optical and Medical FibersAdsys Controls Inc.4:00-4:30 Breaking New Ground with Vision Inspection SystemsColeman Technologies Inc.4:45-5:15 Development of a Digitally Multiplexed Bioassay Reader with Magnetic Bead TechnologyMoviMED4:15-4:45

27