1. CSE 291 Final Project: Adaptive Multi-Spectral Differencing Andrew Cosand UCSD CVRR

2. Differencing Detect changes in a sequence of images. Pixels of reference image are subtracted from the current image to determine how different they are. Pixels with exceed some difference threshold are assumed to correspond to different objects in the images.

3. Differencing Reference Image – Current Image = Difference

4. Problems Differences other than the object of interest may show up. –Pixel noise –Moving background objects (trees, water) –Lighting changes –Camera movement (small) –Shadows & Reflections

5. Pixel Noise

6. Solutions Variations can be included in a background model. –Reference frame may use eg Gaussian mixture models to characterize pixels –Reference frame can be updated at different rates. Very slow basically detects changes from when the system was started, very fast detects changes from the previous frame.

7. Camera Movement

8. Solutions Very small camera movements can be modeled in the background similar to pixel noise or moving background objects Other segmentation methods can be used to identify and track objects in the scene Camera motion can be identified and corrected (Optical flow, correspondence)

9. Shadows Shadow Detected Difference Good Bad

10. Solutions Color Space Conversion –Transform data into more useful form, eg normalized chromaticity or Hue Saturation Intensity colorspace, which separates color and intensity for robust detection in the presence of shadows.

11. HSI Hue angle determines color Saturation determines how ‘colorful’ or ‘washed out’ Intensity determines brightness

12. HIS Colorspace Detection Shadows simply decrease intensity without effecting hue Hue differencing is therefore quite robust to the presence of shadows Great But….

13. Hue Determination To decide what ‘color’ a pixel is, it must first have a ‘color’ Conversion –Normalize R,G,B s.t. 0  r,g,b  1 –h = acos (r-g)+(r-b) 2[(r-g) 2 + (r-b)(g-b)] 1/2 –Very sensitive when r  g  b

14. Hue Differencing Hue ‘Noise’ Causes False Detects

15. Idea Since hue information is unreliable for grayish pixels, ignore hue difference results at these pixels and use intensity instead. Need some weighting function which determines how to do this.

16. Previous Solution Francois and Medioni used a saturation threshold to ignore hue information for gray pixels – Works well –Requires threshold to be set

17. Goal Want a weighting function which will specify a combination of hue and intensity differencing. –Intensity should receive more weight when hue is unreliable –Hue should receive more weight when it can be reliably determined Hope to find some underlying relationship

18. Implementation Using Euclidian distance to gray line as a color measure –Saturation is somewhat tricky (a la Matlab) Ideal system would determine weighting function based on training data, similar to backpropogation

19. Backpropogation Outputs are weighted combinations of inputs Determine errors at outputs Determine how much each input was responsible for the error Adjust each weight accordingly

20. Current Algorithm Examines each pixel, changes weight in proportion to the error –For pixels which should have detected, weight is increased proportionally to 1-detection –For pixels which should NOT have detected, weight is DECREASED proportionally to detection

21. Insights Examination of hue errors shows a definite correlation to coloration

22. Results Weighting Functions

23. Lack of Colorful Data

24. Results Combined Detection

25. Problems Correlation can vary widely from image to image. Weights are noisy, skewed by lack of colorful data Probably needs more data processing No good model determined yet

26. Conclusion System shows definite promise Model still needs to be determined and adaptively fit

27. Shadow Supression

28. References A.R.J. Francois, G.G. Medioni, Adaptive Color Background Modeling for Real-Time Segmentation of Video Streams A. Prati, I. Mikic, M. Trivedi, R. Cucchiara, Detecting Moving Shadows: Formulation, Algorithms and Evaluation T. Horprasert, D. Harwood, L.S. Davis, A statistical approach for real-time robust background subtraction and shadow detection