Shadow Detection and Removal

Slides:



Advertisements
Similar presentations
Change Detection C. Stauffer and W.E.L. Grimson, “Learning patterns of activity using real time tracking,” IEEE Trans. On PAMI, 22(8): , Aug 2000.
Advertisements

CS Spring 2009 CS 414 – Multimedia Systems Design Lecture 4 – Digital Image Representation Klara Nahrstedt Spring 2009.
Robust statistical method for background extraction in image segmentation Doug Keen March 29, 2001.
July 27, 2002 Image Processing for K.R. Precision1 Image Processing Training Lecture 1 by Suthep Madarasmi, Ph.D. Assistant Professor Department of Computer.
Color spaces CIE - RGB space. HSV - space. CIE - XYZ space.
COLORCOLOR A SET OF CODES GENERATED BY THE BRAİN How do you quantify? How do you use?
Color Image Processing
Rob Fergus Courant Institute of Mathematical Sciences New York University A Variational Approach to Blind Image Deconvolution.
December 5, 2013Computer Vision Lecture 20: Hidden Markov Models/Depth 1 Stereo Vision Due to the limited resolution of images, increasing the baseline.
ECCV 2002 Removing Shadows From Images G. D. Finlayson 1, S.D. Hordley 1 & M.S. Drew 2 1 School of Information Systems, University of East Anglia, UK 2.
Shadow removal algorithms Shadow removal seminar Pavel Knur.
Motion Detection And Analysis Michael Knowles Tuesday 13 th January 2004.
Recovering Intrinsic Images from a Single Image 28/12/05 Dagan Aviv Shadows Removal Seminar.
Announcements Big mistake on hint in problem 1 (I’m very sorry).
Segmentation Divide the image into segments. Each segment:
Robust Object Segmentation Using Adaptive Thresholding Xiaxi Huang and Nikolaos V. Boulgouris International Conference on Image Processing 2007.
Image Forgery Detection by Gamma Correction Differences.
Object Detection and Tracking Mike Knowles 11 th January 2005
CS 223B Assignment 1 Help Session Dan Maynes-Aminzade.
CS 376b Introduction to Computer Vision 04 / 01 / 2008 Instructor: Michael Eckmann.
CSE 291 Final Project: Adaptive Multi-Spectral Differencing Andrew Cosand UCSD CVRR.
A Self-Organizing Approach to Background Subtraction for Visual Surveillance Applications Lucia Maddalena and Alfredo Petrosino, Senior Member, IEEE.
Deriving Intrinsic Images from Image Sequences Mohit Gupta Yair Weiss.
Illumination Normalization with Time-Dependent Intrinsic Images for Video Surveillance Yasuyuki Matsushita, Member, IEEE, Ko Nishino, Member, IEEE, Katsushi.
Shadow Removal Seminar
CSE 803 Stockman Fall Color Used heavily in human vision Color is a pixel property, making some recognition problems easy Visible spectrum for humans.
Object recognition under varying illumination. Lighting changes objects appearance.
Introduction of the intrinsic image. Intrinsic Images The method of Finlayson & Hordley ( 2001 ) Two assumptions 1. the camera ’ s sensors are sufficiently.
Shadow Detection In Video Submitted by: Hisham Abu saleh.
Jacinto C. Nascimento, Member, IEEE, and Jorge S. Marques
Real Time Abnormal Motion Detection in Surveillance Video Nahum Kiryati Tammy Riklin Raviv Yan Ivanchenko Shay Rochel Vision and Image Analysis Laboratory.
VINCENT URIAS, CURTIS HASH Detection of Humans in Images Using Skin-tone Analysis and Face Detection.
Optical flow (motion vector) computation Course: Computer Graphics and Image Processing Semester:Fall 2002 Presenter:Nilesh Ghubade
Neighborhood Operations
Tricolor Attenuation Model for Shadow Detection. INTRODUCTION Shadows may cause some undesirable problems in many computer vision and image analysis tasks,
ECE 8433: Statistical Signal Processing Detection of Uncovered Background and Moving Pixels Detection of Uncovered Background & Moving Pixels Presented.
Deriving Intrinsic Images from Image Sequences Mohit Gupta 04/21/2006 Advanced Perception Yair Weiss.
Digital Image Processing (DIP) Lecture # 5 Dr. Abdul Basit Siddiqui Assistant Professor-FURC 1FURC-BCSE7.
December 9, 2014Computer Vision Lecture 23: Motion Analysis 1 Now we will talk about… Motion Analysis.
Motion Analysis using Optical flow CIS750 Presentation Student: Wan Wang Prof: Longin Jan Latecki Spring 2003 CIS Dept of Temple.
1 Motion Analysis using Optical flow CIS601 Longin Jan Latecki Fall 2003 CIS Dept of Temple University.
Autonomous Robots Vision © Manfred Huber 2014.
Jack Pinches INFO410 & INFO350 S INFORMATION SCIENCE Computer Vision I.
1 Machine Vision. 2 VISION the most powerful sense.
Digital Image Processing
CS 376b Introduction to Computer Vision 03 / 31 / 2008 Instructor: Michael Eckmann.
Ec2029 digital image processing
Automatic Caption Localization in Compressed Video By Yu Zhong, Hongjiang Zhang, and Anil K. Jain, Fellow, IEEE IEEE Transactions on Pattern Analysis and.
Learning and Removing Cast Shadows through a Multidistribution Approach Nicolas Martel-Brisson, Andre Zaccarin IEEE TRANSACTIONS ON PATTERN ANALYSIS AND.
Detecting Moving Objects, Ghosts, and Shadows in Video Streams
Video object segmentation and its salient motion detection using adaptive background generation Kim, T.K.; Im, J.H.; Paik, J.K.;  Electronics Letters 
- photometric aspects of image formation gray level images
Color Image Processing
Color Image Processing
Ido Omer Michael Werman
Automatic Video Shot Detection from MPEG Bit Stream
Presenter: Ibrahim A. Zedan
Traffic Sign Recognition Using Discriminative Local Features Andrzej Ruta, Yongmin Li, Xiaohui Liu School of Information Systems, Computing and Mathematics.
Color Image Processing
Image Segmentation – Edge Detection
COMP 9517 Computer Vision Motion 7/21/2018 COMP 9517 S2, 2012.
Motion Detection And Analysis
Introduction to Computer and Human Vision
Range Imaging Through Triangulation
Image Segmentation Techniques
Implementation on video object segmentation algorithm
Color Image Processing
A Novel Smoke Detection Method Using Support Vector Machine
DIGITAL IMAGE PROCESSING Elective 3 (5th Sem.)
Presentation transcript:

Shadow Detection and Removal Image Processing Seminar Shadow Detection and Removal Using Video Advisor: Dr. Hagit Hel-Or By: Ido Yerushalmy February 2008

Using JPEG Characteristics Shadow From Video Using JPEG Characteristics Why remove a shadow? The problem: shadows, like real objects, usually differ from their surroundings and move with the object creating them – making it difficult to distinguish between the 2 types Removing the shadows can help perform segmentation, pattern matching and moving objects extraction Knowing the luminance of the scene allows adding CG sections in a realistic manner JPEG Format Incentives Quant. Table Shadow Characteristics MPEG Format Survey GOP correlation ML Algorithm Double Quantization ML enhancement Summary Summary

Shadow From Video Incentives Types of shadows A cast shadow is the area projected on the scene by an object. It can be divided into: Umbra – direct light is totally blocked Penumbra – light partially blocked Shadow Characteristics Survey ML Algorithm ML enhancement Summary

Using JPEG Characteristics Shadow From Video Using JPEG Characteristics Modeling appearance of a point Incentives For simplicity, assume all surfaces are matt (Lambertian world) Each point we see is the result of its reflectance and the amount of luminance it receives In a mathematical notation: I(x,y) = R(x,y) * L(x,y) Shadow Characteristics Quant. Table MPEG Format Survey GOP correlation ML Algorithm Double Quantization ML enhancement Summary Summary Image = Reflectance x Luminance

Modeling appearance of a point– Cont. Using JPEG Characteristics Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Modeling appearance of a point– Cont. Incentives We can furthermore model the umbra and penumbra cases: Shadow Characteristics Quant. Table Quant. Table Quant. Table MPEG Format MPEG Format MPEG Format Survey GOP correlation GOP correlation GOP correlation ML Algorithm Double Quantization Double Quantization Double Quantization ML enhancement Illuminate Summary Summary Summary Summary Penumbra Umbra Where: CA – ambient light intensity CP – light source intensity L – direction of light source N(x,y) – object surface normal K(x,y) – softening due to penumbra

Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Survey of common methods Incentives Algorithms assume the following: Light source is white, spread evenly and strong The reference image is assumed static, textured and planar Shadow Characteristics Quant. Table Quant. Table Survey MPEG Format MPEG Format GOP correlation GOP correlation ML Algorithm Double Quantization Double Quantization ML enhancement Summary Summary Summary

Statistical non-parametric algorithms Using JPEG Characteristics Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Statistical non-parametric algorithms Incentives Such statistical algorithms assume that shadows have similar chromaticity but lower brightness than the background model. A statistical learning procedure is used to automatically determine the appropriate thresholds. Shadow Characteristics Quant. Table Quant. Table Quant. Table Survey MPEG Format MPEG Format MPEG Format GOP correlation GOP correlation GOP correlation ML Algorithm Double Quantization Double Quantization Double Quantization ML enhancement Summary Summary Summary Summary

Deterministic non-model based with color exploitation Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Using JPEG Characteristics Deterministic non-model based with color exploitation Incentives Shadow Characteristics Quant. Table Quant. Table Quant. Table Survey MPEG Format MPEG Format MPEG Format Uses the HSV color representation GOP correlation GOP correlation GOP correlation ML Algorithm Double Quantization Double Quantization Double Quantization ML enhancement Summary Summary Summary Summary

Deterministic non-model based with color exploitation – cont. Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Using JPEG Characteristics Deterministic non-model based with color exploitation – cont. Incentives Shadow Characteristics Quant. Table Quant. Table Quant. Table Survey MPEG Format MPEG Format MPEG Format A shadow cast does not change the Hue significantly Shadows often lower the saturation and value of the points GOP correlation GOP correlation GOP correlation ML Algorithm Double Quantization Double Quantization Double Quantization ML enhancement Summary Summary Summary Summary

Deterministic non-model based with color exploitation – cont. Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Using JPEG Characteristics Deterministic non-model based with color exploitation – cont. Incentives Shadow Characteristics Quant. Table Quant. Table Quant. Table Survey MPEG Format MPEG Format MPEG Format Formally, it can be described as: GOP correlation GOP correlation GOP correlation ML Algorithm Double Quantization Double Quantization Double Quantization ML enhancement Summary Summary Summary Summary Where: I and B are the input image and its computed background Alpha defines the “power” of the shadow and Beta is used only to avoid misclassifications due to noise

Deterministic non-model based with spatial redundancy exploitation Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Using JPEG Characteristics Deterministic non-model based with spatial redundancy exploitation Incentives Shadow Characteristics Quant. Table Quant. Table Quant. Table Survey MPEG Format MPEG Format MPEG Format Three criteria are tested to detect shadows: The presence of a "darker" uniform regions hints a cast shadow Presence of high difference in luminance with relation to the reference frame The presence of static and moving edges. Static edges hint static shadows. Looking at the width of the edges can reveal if this is a penumbra shadow, since it usually have a softer (= wider) edge Drawback: This method is not enough generic and has too many assumptions GOP correlation GOP correlation GOP correlation ML Algorithm Double Quantization Double Quantization Double Quantization ML enhancement Summary Summary Summary Summary

Maximum Likelihood Algorithm Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Using JPEG Characteristics Maximum Likelihood Algorithm Incentives Shadow Characteristics Assumptions on the world: Lambertian (all surfaces are matt) A single reflectance image (static camera), and many illuminations images. Fits a surveillance camera model Formally denoted by: Quant. Table Quant. Table Quant. Table Survey MPEG Format MPEG Format MPEG Format ML Algorithm GOP correlation GOP correlation GOP correlation Double Quantization Double Quantization Double Quantization ML enhancement Summary Summary Summary Summary For simplicity reasons we will work in the log domain:

Maximum Likelihood Algorithm - cont Using JPEG Characteristics Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Maximum Likelihood Algorithm - cont Incentives Statistical knowledge from previous works: When derivative filters are applied to natural images, the filter output tends to be sparse Shadow Characteristics Quant. Table Quant. Table Quant. Table Survey MPEG Format MPEG Format MPEG Format ML Algorithm GOP correlation GOP correlation GOP correlation Double Quantization Double Quantization Double Quantization ML enhancement Natural Image Horizontal Derivative Hist. Summary Summary Summary Summary

Maximum Likelihood Algorithm - cont Using JPEG Characteristics Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Maximum Likelihood Algorithm - cont Incentives Shadow Characteristics Quant. Table Quant. Table Quant. Table What is this histogram similar to? Survey MPEG Format MPEG Format MPEG Format Horizontal Derivative Hist. Laplace ML Algorithm GOP correlation GOP correlation GOP correlation Double Quantization Double Quantization Double Quantization ML enhancement Summary Summary Summary Summary Given an image, lets try to find its reflectance part by fitting the derivative’s histogram to the expected Laplace. This can be done by taking the pixel wise median over several frames (Maximum Likelihood algorithm)

Maximum Likelihood Algorithm - cont Using JPEG Characteristics Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Maximum Likelihood Algorithm - cont Incentives Shadow Characteristics Quant. Table Quant. Table Quant. Table In a formal manner: Survey MPEG Format MPEG Format MPEG Format ML Algorithm GOP correlation GOP correlation GOP correlation Where: On – is the nth derivative filter The median is for each pixel over a series of frames Note: this is true as long as the reflectance image is indeed constant Double Quantization Double Quantization Double Quantization ML enhancement Summary Summary Summary Summary Retrieving the luminance image is now easy: l(x,y,t) = i(x,y,t)-r(x,y)

Maximum Likelihood Algorithm - cont Shadow From Video Using JPEG Characteristics Using JPEG Characteristics Using JPEG Characteristics Maximum Likelihood Algorithm - cont Incentives Shadow Characteristics Quant. Table Quant. Table Quant. Table Retrieving the reflectance (and luminance) images: Survey MPEG Format MPEG Format MPEG Format ML Algorithm GOP correlation GOP correlation GOP correlation Double Quantization Double Quantization Double Quantization ML enhancement Summary Summary Summary Summary

Maximum Likelihood Algorithm - cont Using JPEG Characteristics Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Maximum Likelihood Algorithm - cont Incentives Shadow Characteristics Quant. Table Quant. Table Quant. Table Another example: Survey MPEG Format MPEG Format MPEG Format Frame 2 Frame 11 ML reflectance ML illumination 2 ML illumination 11 ML Algorithm GOP correlation GOP correlation GOP correlation Double Quantization Double Quantization Double Quantization ML enhancement Summary Summary Summary Summary How do you produce such a sequence of images?

Maximum Likelihood Algorithm - cont Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Using JPEG Characteristics Maximum Likelihood Algorithm - cont Incentives Shadow Characteristics A common way is to use a device like this: Quant. Table Quant. Table Quant. Table Survey MPEG Format MPEG Format MPEG Format ML Algorithm GOP correlation GOP correlation GOP correlation Double Quantization Double Quantization Double Quantization ML enhancement Summary Summary Summary Summary

ML enhancement This algorithm is based on the one we just explored Using JPEG Characteristics Using JPEG Characteristics Using JPEG Characteristics Shadow From Video ML enhancement Incentives This algorithm is based on the one we just explored Uses a static camera view (same as before) Does not assume a Lambertian world, resulting in a changing reflectance image. Shadow Characteristics Quant. Table Quant. Table Quant. Table Survey MPEG Format MPEG Format MPEG Format ML Algorithm GOP correlation GOP correlation GOP correlation ML Enhancement Double Quantization Double Quantization Double Quantization Summary Summary Summary Summary

ML enhancement For example: Using JPEG Characteristics Using JPEG Characteristics Using JPEG Characteristics Shadow From Video ML enhancement Incentives For example: The difference in reflectance at the border between white stripes and a black road is not constant, since the white stripes change their reflectance as a factor of time (light direction). This causes such areas to be considered (falsely) as part of the illumination image. Shadow Characteristics Quant. Table Quant. Table Quant. Table Survey MPEG Format MPEG Format MPEG Format ML Algorithm GOP correlation GOP correlation GOP correlation ML Enhancement Double Quantization Double Quantization Double Quantization Summary Summary Summary Summary

Using JPEG Characteristics Shadow From Video Using JPEG Characteristics ML enhancement – cont. Incentives Shadow Characteristics Note that the problematic areas are where textures exist Overcoming the problem: Start by generating a basic (ML) reflectance image Try to move the textures from the illumination image to the reflectance image. One way to do it is: Quant. Table Quant. Table Quant. Table Survey MPEG Format MPEG Format MPEG Format ML Algorithm GOP correlation GOP correlation GOP correlation ML Enhancement Double Quantization Double Quantization Double Quantization Summary Summary Summary Summary

ML enhancement – cont. This (almost) solves the problem we saw before: Using JPEG Characteristics Using JPEG Characteristics Shadow From Video Using JPEG Characteristics ML enhancement – cont. Incentives Shadow Characteristics This (almost) solves the problem we saw before: Quant. Table Quant. Table Quant. Table Survey MPEG Format MPEG Format MPEG Format ML Algorithm GOP correlation GOP correlation GOP correlation ML Enhancement Double Quantization Double Quantization Double Quantization Summary Summary Summary Summary

Using JPEG Characteristics Shadow From Video Using JPEG Characteristics Using JPEG Characteristics Using JPEG Characteristics Summary Incentives JPEG Format Shadow Characteristics Quant. Table Quant. Table Quant. Table Quant. Table Shadow removal can be helpful as a pre-processing stage in segmentation and pattern matching algorithms Shadow modeling is difficult. Some assumptions must be made. Some of the assumptions introduced: Lambertian world, natural image statistics, constant reflectance image … Assuming a changing reflectance gives better result, but relies on additional assumptions Survey ML Algorithm ML enhancement Summary P-frames are not discussed here, since a different encoder may also generate a different motion vector – causing the P frames to look completely different (so the double-quantization is undetectable)

Using JPEG Characteristics References JPEG Format Quant. Table A. Prati, R. Cucchiara, I. Mikic, M.M. Trivedi, "Analysis and detection of shadows in video streams: a comparative evaluation" in Proceedings of Computer Vision and Pattern Recognition conference (CVPR 2001), vol. 2, pp. 571-576, 2001 Y. Weiss. “Deriving intrinsic images from image sequences”. In ICCV01, pages II: 68-75. IEEE, 2001 Y. Matsushita and K. Nishino, “Illumination normalization with time-dependent intrinsic images for video surveillance,” IEEE Trans. Pattern Anal. Mach. Intell., 26, pp. 1336-1348 (2004) Athinodoros S. Georghiades Peter N. Belhumeur, “From Few to Many: Illumination Cone Models for Face Recognition Under Variable Lighting and Pose”, IEEE Trans. on Pattern Analysis and Machine Intelligence MPEG Format GOP correlation Double Quantization Summary P-frames are not discussed here, since a different encoder may also generate a different motion vector – causing the P frames to look completely different (so the double-quantization is undetectable)