1. Shadow removal algorithms Shadow removal seminar Pavel Knur

2. Deriving intrinsic images from image sequences Yair Weiss July 2001.

3. History “ intrinsic images ” by Barrow and Tenenbaum, 1978

4. Constraints Fixed viewpoint Works only for static objects Cast shadows

5. Classic ill-posed problem Denote – the input image – the reflectance image – the illumination image Number of Unknowns is twice the number of equations.

6. The problem Given a sequence of T images in which reflectance is constant over the time and only the illumination changes, can we solve for a single reflectance image and T illumination images ? Still completely ill-posed : at every pixel there are T equations and T+1 unknowns.

7. Maximum-likelihood estimation Log domain :

8. Assumptions When derivative filters are applied to natural images, the filter outputs tend to be sparse.

9. Laplacian distribution Can be well fit by laplacian distribution

10. Claim 1 Denote : N filters – Filter outputs – Filtered reflectance image – ML estimation of filtered reflectance image is given by

11. Estimated reflectance function Recover ML estimation of r is reversed filter of

12. ML estimation algorithm

13. ML estimation algorithm – cont. Ones we have estimated

14. Claim 2 What if does not have exactly a Laplasian distribution ? Let Then estimated filtered reflectance are within with probability at least:

15. Claim 2 - proof If more than 50% of the samples of are within of some value, then by definition of median, the median must be within of that value.

16. Example 1 Einstein image is translated diagonally 4 pixels per frame

17. Example 2 64 images with variable lighting from Yale Face Database

18. Illumination Normalization with Time- Dependent Intrinsic Images for Video Surveillance Y.Matsushita,K.Nishito,K.Ikeuchi Oct. 2004

19. Illumination Normalization algorithm Preprocessing stage for robust video surveillance. Causes –Illumination conditions –Weather conditions –Large buildings and trees Goal –To “ normalize ” the input image sequence in terms of incident lighting.

20. Constraints Fixed viewpoint Works only for static objects Cast shadows

21. Background images Remove moving objects from the input image sequence Input images Background images Off-line

22. Estimation of Intrinsic Images Denote input image time-varying reflectance image time-varying illumination image reflectance image estimated by ML illumination image estimated by ML Filters Log domain Input images Background images Off-line Estimation of Intrinsic Images

23. Estimation of Intrinsic Images – cont. In Weiss ’ s original work The goal is to find estimation of and Input images Background images Off-line Estimation of Intrinsic Images

24. Estimation of Intrinsic Images – cont. Basic idea: Estimate time-varying reflectance components by canceling the scene texture from initial illumination images Define: Input images Background images Off-line Estimation of Intrinsic Images

25. Estimation of Intrinsic Images – cont. Finally : Where : is reversed filter of Input images Background images Off-line Estimation of Intrinsic Images

26. Shadow Removal Denote - background image - illuminance-invariant image Input images Background images Off-line Estimation of Intrinsic Images

27. Illumination Eigenspace PCA – Principle component analysis Basic components - Input images Background images Off-line Estimation of Intrinsic Images Illumination Eigenspace

28. Illumination Eigenspace – cont. Average is P is MxN matrix where –N – number of pixels in illumination image –M – number of illumination images Covariance matrix Q of P is Input images Background images Off-line Estimation of Intrinsic Images Illumination Eigenspace

29. Direct Estimation of Illumination Images Pseudoillumination image Direct Estimation is Where –F is a projection function onto the j ’ s eigenvector - Input images Background images Off-line Estimation of Intrinsic Images Illumination Eigenspace

30. Direct Estimation of Illumination Images Results Input images Background images Off-line Estimation of Intrinsic Images Illumination Eigenspace

31. Shadow interpolation probability density function cumulative probability function shadowed area lit area mean optimum threshold value Input images Background images Off-line Estimation of Intrinsic Images Illumination Eigenspace Shadow Interpolation

32. The whole algorithm Input images Background images Off-line Estimation of Intrinsic Images Illumination Eigenspace / Illumination Images Normalization Shadow Interpolation

33. Example

34. Questions ?

35. References [1] Y.Weiss, ” Deriving Intrinsic Images from Image Sequences ”, Proc. Ninth IEEE Int ’ l Conf. Computer Vision, pp. 68-75, July 2001. [2] Y.Matsushita,K.Nishito,K.Ikeuchi, “ Illum ination Normalization with Time- Dependent Intrinsic Images for Video Surveillance ”,Oct. 2004.