1. California Car License Plate Recognition System ZhengHui Hu Advisor: Dr. Kang

2. 12/11/2006License Plate Recognition System2 Introduction A License Plate Recognition System (LPRS) is a system to automatically detect, recognize and identify a vehicle plate. It involves low-level image processing techniques with higher level artificial intelligence techniques.

3. 12/11/2006License Plate Recognition System3 Applications Mainly for monitoring, surveillance and security. For example, – Entrance/Exit monitoring for parking lot structures – Part of surveillance system for gated communities – Control gateways for vehicle passage – Security Systems for high traffic Law Enforcement

4. 12/11/2006License Plate Recognition System4 Technical Issues Image Capturing – Vehicle speed – Lighting condition – Occlusion Processing speed – Heavy traffic Recognition accuracy – High correctness

5. 12/11/2006License Plate Recognition System5 Current State There are many companies, especially in Europe, that developed this type of system commercially There are many research trying to improve accuracy and speed performance

6. 12/11/2006License Plate Recognition System6 System Architecture Plate Region Segmentation – Locate plate region out of car and/or background Character Segmentation – Segment each character/number out of plate Character Recognition – Recognize each character on the plate – Similar to OCR process Plate Segmentation Module Character Segmentation Module Character Recognition Module

7. 12/11/2006License Plate Recognition System7 Previous Work Most previous work are focused on the character segmentation and recognition process based on – Fuzzy algorithms – Template matching – Neural network

8. 12/11/2006License Plate Recognition System8 System Development System is developed in C++ – OpenCV library is used for image processing and display Development Environment – Microsoft Visual Studio 2003 was used for development under Windows – Eclipse CDT was used under LINUX

9. 12/11/2006License Plate Recognition System9 Step1 - Plate Region Segmentation Goal – Locate license plate in an image Target image group – California Car License Plates (regular ones) Challenges – Location: plate regions at random place – Size: vehicle distance from the camera affect plate size – Color: affected by lighting conditions (day/night/shadow) – Skew/distortion: images can be taken from different angles

10. 12/11/2006License Plate Recognition System10 Step1 - Plate Region Segmentation Helpful information – All License Plate have same shape – Known background/foreground colors Light background color Bluish foreground color –numbers and characters – Color distribution in a rectangular plate region

11. 12/11/2006License Plate Recognition System11 Step1 - Plate Region Segmentation Image Preprocessing Filter using Color and Edge Information Connected Components Analysis Edge Information Find Candidate Regions Input Image Feedback for more filtering

12. 12/11/2006License Plate Recognition System12 Input Images Captured using a digital camera – Different distance – Different lighting conditions – Different angles Original size 2048X1536 Resized to 800X600 for faster process

13. 12/11/2006License Plate Recognition System13 Input Image Preprocessing Two steps performed – Gaussian filter for noise elimination – Brightness normalization through histogram equalization

14. 12/11/2006License Plate Recognition System14 Edge Information Apply morphological operator to detect region of high change. Plate character/numbers are among these

15. 12/11/2006License Plate Recognition System15 Filter Filter using Color and Edge Information – Use edge information to find plate background color – Filter image using plate background color

16. 12/11/2006License Plate Recognition System16 Connected Component Analysis Find connected component and values – Width/Height ratio – Amount of edge pixels

17. 12/11/2006License Plate Recognition System17 Find Candidate Plate has ratio between 1 and 3 Plate has highest or 2 nd highest pixel density from edge image

18. 12/11/2006License Plate Recognition System18 Experiment Results Total Pictures Tested: 43 – Region found: 38 – Region not found: 5 – Success rate: 88% Error classification – Filtering process chopped out part of plate – Fail to identify correct candidate region

19. 12/11/2006License Plate Recognition System19 Experiment Results (Speed) Machine Used for Testing: Pentium 4-M 1.70Ghz, 256 MB RAM – For images 800X600, the processing time is 150 ~ 190 ms – For original size image 2048X1536, processing time is around 1 sec

20. 12/11/2006License Plate Recognition System20 Step2 - Character Segmentation Segment each character/number out of the plate detected by previous module Challenges – Rectangle segmented might contain more than just the plate – Plate might contain some things other than number/characters Still under development

21. 12/11/2006License Plate Recognition System21 Step3 - Character Recognition A process to recognize each character/number segmented Challenges – Noise – Image scaling and distortion – Image corruption

22. 12/11/2006License Plate Recognition System22 Step3 - Character Recognition Our approach – Artificial Neural networks are used to recognize characters and digits – During training process, simulated annealing process was added to the back propagation training to avoid the problem of local minimal Still under development

23. 12/11/2006License Plate Recognition System23 Conclusion Contributions – Algorithm for plate detection – Combination of back-propagation/simulated annealing process in neural network training Future Work – Improve recognition ratio in step1 via feedback for filtering and better connected component analysis – Finish Step3

24. 12/11/2006License Plate Recognition System24 References See Resources page in Website for full list of referencesWebsite