Reading 1D Barcodes with Mobile Phones Using Deformable Templates

1D BARCODES  Optical machine readable representation of data relating to the objects.  Represented data by varying the widths and spacings of parallel lines.  Referred as linear or one-dimensional (1D).

Reading with Using 1D Barcodes Mobile Phones Deformable Templates

MOBILE PHONES  A device that can make and receive telephone calls.  Large variety of applications.

Reading with Using 1D Barcodes Mobile Phones Deformable Templates

DEFORMABLE TEMPLATES  Represent each digit of the barcode, averaging over the set of all expected deformations.  The success depends on an accurate description of the shape class.

BARCODE READERS 1)Pen type readers  Consist of a light source and photodiode that are placed next to each other in the tip of a pen. 2) Laser based Scanners  Laser scanners work the same way as pen type readers except that they use a laser beam as the light source.

3) Hand held Barcode Scanners  This consists of a light source, a lens and a light sensor translating optical impulses into electrical ones. 4) Regular Mobile Phones with Camera  Cell phone cameras taking the images of bar there are 2D barcodes which are optimized for cell phones.

5)Mobile Phones using Deformable Templates  In this method, for barcode decoding (localization and reading) that can deal with images that are blurred, noisy, and with low resolution.

BARCODE READING PROCEDURE In early:- 1) Binarization Converting images to black and white image. 2) Edge Extraction Extraction of barcode edges. In this approach:- 1)Localization Identify the location of the endpoints of the barcode in the image. 2)Decoding Converting the black and while strips in barcodes to numbers.

THIS APPROACH Assumptions:  No Binarization  No Edge Extraction  Used in noise and blur images  Not sensitive  Gray level information is used  Optimization Procedure  Use particular forms of deformable Templates  2 Operations:- 1) LOCALIZATION 2) DECODING

1) BARCODE LOCALIZATION  Simple method  Image captured from mobile Camera  Vertical Axis is Parallel to Bars  Scanline determination

STEPS IN LOCALIZATION PROCESS Original image (1,152 x864, JPEG compressed). 1) Smoothened Map calculation(Is(n)) Horizontal gradient is Strong and denoted by Ix(n) Vertical gradient is weak and denoted by Iy(n) Ie(n)=|Ix(n)|-|Iy(n)| where n=pixel

- Smoothened map calculation(Is(n)) Run a block filter of size 31x31 over Ie(n) we get smoothened map. The smoothed map Is(n) with its maximum marked by a black square

2)Binarization of Is(n) Binarization is done using single threshold. Binarization by thresholding of Is(n) 3)Scanline identification  Selection of pixel and rectangle drawing -Select the pixel n0 that maximizes Is(n) -Expand a vertical and a horizontal line from n0, and form a rectangle  Scanline identification The horizontal line l(n) that passes through the center of this rectangle is chosen as the scanline for the analysis.  Localize end points First determine the intersections iL and iR of the scanline l(n) with the rectangle.

 Final localized image -The rectangle being larger than the actual barcode, we proceed inward from each end. The resulting rectangular segment (black square) *Horizontal scanlines are only considered -Barcode slanted less than 45 angle succeds

BARCODE DECODING -Analysis is based on single scanline extracted from detected barcode area. -Beginning and end points of the barcode pattern in the scanlines are detected with certain accuracy.

Spacial location computation Intensity values comparing Deformable template Symbol generation STEPS IN DECODING A

A Global cost function Reading decoded value Position detection Model creation

STEPS IN BARCODE DECODING -Computation of Spacial location Compute the spatial location of each digit segment in the barcode. -Comparing intensity values Compare the intensity profile of the corresponding segment of the scanline with binary templates, each representing a symbol. -Deformable templates Measure how well a deformed (shifted and scaled) template explains the observed intensity. -Symbol generation For each digit segment produce a sequence of symbols.

-Global cost function This is to avoid the problems caused by blur, low resolution and noise. This function can be found using Dynamic Programming. -Model creation A model (or template)Mk is defined for a given symbol k as a continuous piecewise constant function that alternates between -1 and 1, where a value of -1 (1) represents a black (white) bar. -Position detection The approximate position of each digit segment of the barcode is computed.

-Reading decoded value To read the value of the digit, we simply compare the intensity I(n) within the segment with the models and pick the model that best fits the data. Length of the barcode is equal to 95 times the base width, and the first three bars are guard bars. Dynamic programming can be used to minimize the cost function C over the space of sequences.

IMPLEMENTATIONS AND TESTING  Matlab  Symbian OS MATLAB IMPLEMENTATION Localization: Assumptions: -User is holding the cell phone -Bars are vertical in the image -Produces good results when the phone is rotated at an angle. Localization comprises two steps: *Segmentation of the barcode image *Determination of the horizontal scanline segment used for decoding.

The localization algorithm assumes that the cell phone is held in a vertical fashion. However, it proves robust to a large deviation from the ideal setup. An image of a barcode at 37 degreesComputed energy map A image of barcode at 45 degrees Computed energy map

Decoding:  Scan lines with end points are taken as input.  Performance of the algorithm is accessed by testing on variety of images. 1) Dataset-1 -Images at high resolution (1,024 x768) -Taken by Nokia N95 cell phone -Autofocus capability 2) Dataset-2 -Images at resolution (640x480) -Taken by Nokia N95 cell phone -Highly compressed in JPEG 3) Dataset-3 -Images at high resolution (1,152x864) -Taken with an older Nokia 7610 cell phone -Fixed focus All three data sets have multiple pictures of several barcodes, taken from different distances and in different conditions.

Example of barcodes correctly decoded by our algorithm from the three data sets The crop outs in the first row are from 1,024 x768 images (data set-1) The crop outs in the second row are from 640 x480 images (data set-2) The crop outs in the third row from 1,152 x864 images (data set 3)  These crop outs are the output of the localization algorithm.  Failure was due to incorrect initial localization.  This decodes barcodes even when the image quality is extremely poor, as can be appreciated by zooming in on the images.

IMPORTANT DETAILS GATHERED 1) The energy in the area corresponding to the barcode is low for angles from 45 degrees and above. 1) Scanline intersect all of the bars, from the left to the right guard bars. 1) The scanline was correctly extracted for angles up to 30 degrees. 1) Manual selection of end points oL and oR and compared them with the output of our localization algorithm. 1) The execution time is only a function of the size of the image.

SYMBIAN IMPLEMENTATION Implemented in Symbian and tested it on a Nokia N95 8 GB device using VGA (640 x480) still images. Nokia N95 cell phone Localization: Localization is same as that in Matlab.

Decoding: Features noticed:  When their occur a failure at reading the barcode, the process can be quickly repeated.  Location or distance of the barcode is not depending, as long as the resolution is sufficient for decoding.  Our implementation can successfully localize and read barcodes that are 2.5 cm wide when they are imaged from up to 30 cm away.  Barcodes that are 4 cm wide can be decoded from up to 45 cm.  Advantage is not to place the phone in close proximity to the barcode

MATLAB SYMBIAN OS LOCALIZATION Segmentation Segmentation Scanline determination Polygon determination DECODING Scanline is taken and Polygon determination anddecode TIME seconds milliseconds PERFORMANCE HighHigh COMPARISON

CONCLUSION By this method we can decode barcode (localization and reading) images that are taken by mobile phones which are blurred, noisy, and with low resolution. Merits  Highly accurate  No binarization is needed  Reduce the risk of errors  Use deformable templates to represent each digit of the barcode  Improved performance  Datasets implemented are given as public  Reading can be done in less than 0.5 seconds(Localization+Decoding)

Demerits  Platform dependent  Apple iOS is not included  Problem occur when phone battery goes off  Handset diversity  Continues video streaming is needed for processing  Pressing keys

FUTURE SCOPE 1) Use in shops for barcode reading 2) Situable for mobile vision applications 3) Easy to handle 4) Reduce the hardware

THANK YOU…!