1. 9/13/2015Memorial University of Newfoundland Faculty of Engineering & Applied Science Engineering 7854 Industrial Machine Vision INTRODUCTION TO MACHINE VISION II Prof. Nick Krouglicof

2. 9/13/2015Memorial University of Newfoundland Presentation Outline 1.Machine vision systems for mechanical metrology Algorithms for camera calibration Development of 3D vision systems for MI Flume Tank 2.Industrial Applications of Machine Vision High speed, line scan camera-based inspection system for the food processing industry Vision based inspection of liquid crystal display (LCD) modules

3. 9/13/2015Memorial University of Newfoundland A systematic approach to the calibration of machine vision systems for industrial metrology In the field of machine vision, camera calibration refers to the experimental determination of a set of parameters which describe the image formation process for a given analytical model of the machine vision system. Ideally, camera calibration is performed without specialized optical equipment, without modifications to the hardware, and without a priori knowledge of the vision system. Most calibration techniques are based on the observation of planar (2D) targets with a large number of control points.

4. 9/13/2015Memorial University of Newfoundland A systematic approach to the calibration of machine vision systems for industrial metrology The machine vision parameters which must be identified include : a)The scale factor b)The frame buffer coordinates of the image center c)The effective focal length of the lens-camera assembly d)The radial lens distortion coefficient e)The pose (position and orientation) of the camera Parameters a) through d) are classified as intrinsic, e) as extrinsic.

5. 9/13/2015Memorial University of Newfoundland A systematic approach to the calibration of machine vision systems for industrial metrology

6. 9/13/2015Memorial University of Newfoundland A systematic approach to the calibration of machine vision systems for industrial metrology

7. 9/13/2015Memorial University of Newfoundland An Efficient Camera Calibration Technique Offering Robustness and Accuracy Over a Wide Range of Lens Distortion

8. 9/13/2015Memorial University of Newfoundland An Efficient Camera Calibration Technique Offering Robustness and Accuracy Over a Wide Range of Lens Distortion CALIBRATION ACCURACY AND THE LENS DISTORTION MODEL

9. 9/13/2015Memorial University of Newfoundland Underwater 3D Vision Systems for MI Flume Tank

10. 9/13/2015Memorial University of Newfoundland Prototype Underwater Stereo Vision System

11. 9/13/2015Memorial University of Newfoundland Calibration Target for Underwater Stereo Vision System

12. 9/13/2015Memorial University of Newfoundland Development of an Intelligent 3D Vision System for Underwater Environment through Actively Manipulated Laser Triangulation

13. 9/13/2015Memorial University of Newfoundland Development of an Intelligent 3D Vision System for Underwater Environment through Actively Manipulated Laser Triangulation

14. 9/13/2015Memorial University of Newfoundland Industrial Machine Vision Applications

15. 9/13/2015Memorial University of Newfoundland Image Analysis Tools for Automated Inspection in the Food Processing Industry: X-Ray Enhancement

16. 9/13/2015Memorial University of Newfoundland Image Analysis Tools for Automated Inspection in the Food Processing Industry: Multispectral Imaging

17. 9/13/2015Memorial University of Newfoundland Image Analysis Tools for Automated Inspection in the Food Processing Industry: Multispectral Imaging

18. 9/13/2015Memorial University of Newfoundland High Speed, Line Scan Based Inspection System

19. 9/13/2015Memorial University of Newfoundland Objective: To remove defects (I.e. visible, dark particle larger than 0.007”) from apple sauce System must be able to handle 12 metric tons per 8 hour shift System must remove 95% of visible defects Line Scan Based Inspection System: Specifications

20. 9/13/2015Memorial University of Newfoundland 2 distinct challenges: Detection Removal High Speed, Line Scan Based Inspection System

21. 9/13/2015Memorial University of Newfoundland High Speed, Line Scan Based Inspection System

22. 9/13/2015Memorial University of Newfoundland High Speed, Line Scan Based Inspection System

23. 9/13/2015Memorial University of Newfoundland Detection system based on a high performance line scan camera; 4096 pixels per line at 4800 lines per second. Image acquisition and processing functions implemented on a Complex Programmable Logic Device (CPLD) as opposed to a microprocessor or Digital Signal Processor (DSP). The objective is to implement image processing functions with dedicated logic gates (i.e. hardware) for real-time performance. Line Scan Based Detection System

24. 9/13/2015Memorial University of Newfoundland What are CPLDs? Complex Programmable Logic Devices (CPLDs) are a class of programmable logic device that are commonly used to implement complex digital designs on a single integrated circuit. Applications of CPLDs in the field of computer engineering include the implementation of bus controllers, address decoders, priority encoder and state machines

25. 9/13/2015Memorial University of Newfoundland Line Scan Camera-Based Detection System

26. 9/13/2015Memorial University of Newfoundland Line Scan Camera-Based Detection System

27. 9/13/2015Memorial University of Newfoundland Line Scan Camera-Based Detection System Typical Particle Typical section of apple sauce recorded with an area scan camera

28. 9/13/2015Memorial University of Newfoundland Line Scan Camera-Based Detection System

29. 9/13/2015Memorial University of Newfoundland Line Scan Camera-Based Detection System

30. 9/13/2015Memorial University of Newfoundland Removal System

31. 9/13/2015Memorial University of Newfoundland Removal System

32. 9/13/2015Memorial University of Newfoundland Removal System: Flow Characterization Rheological nomenclature and associated velocity profiles for steady flow through tubes with circular cross section.

33. 9/13/2015Memorial University of Newfoundland Viscosity Measurement

34. 9/13/2015Memorial University of Newfoundland Viscosity Measurement Velocity profile can be characterized by power law!

35. 9/13/2015Memorial University of Newfoundland Flow Profile of Apple Sauce

36. 9/13/2015Memorial University of Newfoundland Particle Removal Window

37. 9/13/2015Memorial University of Newfoundland System Timing Diagram

38. 9/13/2015Memorial University of Newfoundland Aspiration Valve Characterization

39. 9/13/2015Memorial University of Newfoundland Aspiration Valve Characterization

40. 9/13/2015Memorial University of Newfoundland Particle Detection Rate Versus Flowrate

41. 9/13/2015Memorial University of Newfoundland High Speed, Line Scan Based Inspection System: Current Status Industrial partner is currently developing the production version of the system. Packaging of the principle components (i.e., lenses, cameras, electronics, light sources) remains a major challenge given the environment. One possible solution is to integrate all the electronics in the camera enclosure. Partner is anxious to explore applications in the pulp and paper industry.

42. 9/13/2015Memorial University of Newfoundland Vision Based Inspection of Liquid Crystal Display (LCD) modules Objective: to automate the inspection of LCD modules in order to improve quality control One step in the implementation of a Six-Sigma Program (“3.4 defects per million opportunities”) The inspection must be completed within 30 seconds for 10 predetermined LCD patterns System can “learn” new LCD modules without modifying software

43. 9/13/2015Memorial University of Newfoundland Vision Based Inspection of LCD modules: System Components Pulnix camera with macro lens High frequency fluorescent light sources Coreco Bandit integrated image acquisition and VGA accelerator Software developed using with WiT graphical programming environment in combination with Microsoft VB

44. 9/13/2015Memorial University of Newfoundland Vision Based Inspection of Liquid Crystal Display (LCD) modules

45. 9/13/2015Memorial University of Newfoundland Original Image Showing Error in Alignment Vision Based Inspection of Liquid Crystal Display (LCD) modules

46. 9/13/2015Memorial University of Newfoundland Thresholding Operation – Image Subtraction with respect to an image with no segments illuminated Vision Based Inspection of Liquid Crystal Display (LCD) modules

47. 9/13/2015Memorial University of Newfoundland Blob Analysis – Reference Points are Identified Vision Based Inspection of Liquid Crystal Display (LCD) modules

48. 9/13/2015Memorial University of Newfoundland Image Rotation and Translation Vision Based Inspection of Liquid Crystal Display (LCD) modules

49. 9/13/2015Memorial University of Newfoundland Pixel by Pixel Image Subtraction from Reference Image – Thinning Operator Vision Based Inspection of Liquid Crystal Display (LCD) modules

50. 9/13/2015Memorial University of Newfoundland Blob Analysis Vision Based Inspection of Liquid Crystal Display (LCD) modules

51. 9/13/2015Memorial University of Newfoundland Blob Analysis Vision Based Inspection of Liquid Crystal Display (LCD) modules: Conclusions Inspection system was installed at BAE Systems Canada Ltd. where it was used to test between 200 and 600 LCD displays per day. Number of defective modules that passed inspection was basically reduced to zero. Occasional “false positives” proved to be technical problems with the devices that previously went unreported. Applications for this technology are numerous given the number of LCD displays produced annually.

52. 9/13/2015Memorial University of Newfoundland QUESTIONS ?