1. Robust statistical method for background extraction in image segmentation Doug Keen March 29, 2001

2. Source Paper Rodriguez, Arturo A., Mitchell, O. Robert. “Robust statistical method for background extraction in image segmentation” Stochastic and Neural Methods in Signal Processing, Image Processing, and Computer Vision. Vol. 1569, 1991

3. Problem Given a digital image, how can we differentiate objects of interest from the background?

4. Problem One simple and fast method is thresholding –Create a graytone histogram of a sample of the background –Find threshold values for the right and left shoulders of the background histogram –Compare graytone values of all other pixels of the image to the background histogram If pixel falls between right and left background thresholds, that pixel belongs to the background

5. Problems with thresholding Local background variations may be small, but background variations across the entire image may be substantial Thresholding also assumes just a single background

6. Problems with thresholding In applications with a fixed background, one could do an empirical analysis of a background only image –Once an object is introduced, graytone properties of the background could change (due to reflectance properties of the object, scene illumination, automatic gain control of the camera, and/or shadows)

7. Alternate solution A background extraction method that is based on local statistical measurements performed on log-transformed image data –Works despite smooth changes in background –Doesn’t matter if objects are darker or brighter than the background –Works with multiple backgrounds –Works without a priori knowledge of the background of the image

8. Log-Transformation Log-transformation can help reduce various illumination effects in the image g(x, y) = i(x, y) * f(x, y) g(x,y) is the grayscale value of a pixel i(x,y) is the illumination component f(x,y) is the reflectance component A log-transformation would cause multiplicative illumination effects to become additive

9. Log-Transformation In an image with graytone values: g  [g L, g R ] The log-transformation is expressed as: Where P is a fraction of the original gray scale resolution

10. Background Extraction Segmentation Method Step 1: Decompose the image into a grid of non-overlapping blocks Blocks along the periphery are boundary blocks, and blocks that are not boundary blocks are interior blocks.

11. Step 2: For each block in the image, calculate the mean graytone, the left and right standard deviations of the log-transformed histogram (denoted by , T left, and T right respectively) Background Extraction Segmentation Method

12. Step 2 (cont.): A block can be considered homogenous if Q percent or more of its pixels belong to the same class. Specifically, a block can be considered homogenous if: and Where S is the standard deviation of the image Homogenous blocks can be object-homogenous or background-homogenous. If the block is not homogenous, it is considered uncertain. Background Extraction Segmentation Method

13. Step 2 (cont.): Homogenous boundary blocks are assumed to be background blocks, and their measured statistical parameters are considered their background distributions Background Extraction Segmentation Method

14. Step 3: Non-homogenous boundary blocks are examined by starting from an arbitrary homogenous boundary block and proceeding [counter]clockwise around the boundary blocks The background parameters of a non-homogenous boundary block are estimated from the two nearest background-homogenous blocks Once the background parameters of the non-homogenous boundary block have been estimated, that block is marked background-homogenous Background Extraction Segmentation Method

15. Step 4: Interior blocks are then examined in a certain sequence that assures that the block being examined has 3 adjacent blocks that have already had their background parameters estimated. One of these blocks must be horizontally adjacent, one must be vertically adjacent, and the third must be diagonally adjacent between the two other adjacent blocks. Background Extraction Segmentation Method XX XY XXX Y X XYX

16. Step 4 (cont.): A homogenous interior block is considered object- homogenous if its measured graytone mean and the background mean of a vertically or horizontally adjacent block are significantly different. Otherwise it is background-homogenous. Background parameters of background-homogenous blocks are measured, while background parameters of object-homogenous or uncertain blocks are estimated. Background Extraction Segmentation Method

17. Step 5: Once the background parameters of each block have been measured or estimated, calculate the left and right shoulder thresholds of the background distribution using the following formulas: Where is a prespecified constant (in the case of this study’s experiments, 2.5) Assign these threshold values to the center of each block Background Extraction Segmentation Method

18. Step 5: (dots indicate positioning of threshold values) Background Extraction Segmentation Method          

19. Step 6: Left and right shoulder thresholds for a pixel (x, y) are obtained by bilinear interpolation of the left and right shoulder thresholds assigned to the four block centers surrounding that pixel Let L[g(x,y)] be the log-transformed graytone of pixel (x,y) Pixel (x,y) is darker than the background if L[g(x,y)] < t left Pixel (x,y) is lighter than the background if L[g(x,y)] > t right Background Extraction Segmentation Method

20. Step 6 (cont.): In order to preserve local brightness relationships between object pixels and background pixels, keep two floating histograms as pixels are being classified: one of bright pixels (F b ) and the other of dark pixels (F d ). The abscissa of these floating histograms represents the difference between the log-transformed data value of a pixel and the log-transformed data value of the corresponding background shoulder threshold at that pixel Background Extraction Segmentation Method

21. Step 7: Once every pixel has been classified by the background extraction procedure, calculate the mean and the left and right standard deviations of F b and F d. Calculate left and right shoulder thresholds of F b and F d : Background Extraction Segmentation Method

22. Final pixel classification can then be obtained from: Background Extraction Segmentation Method (Where represents the label associated with each of the descriptive categories)

23. Experimental Results See figures in handout

24. Assessment ProsCons No a priori knowledge req’d Works with non-uniform backgrounds Performs successfully without parameter adjustments or human interaction Works under different lighting conditions Doesn’t matter if the object is darker or brighter than the background… there doesn’t even have to be an object Only tested on industrial scenes and moderately complex outdoor scenes… unproven on natural scenes Estimation of background parameters may be error-prone Only works on grayscale images

25. Possible Improvements Color image support Processing spatial and local information in addition to brightness information to reduce misclassified pixels Robust and real-time performance on natural scenes