When Does a Camera See Rain? Department of Computer Science Columbia University Kshitiz Garg Shree K. Nayar ICCV Conference October 2005, Beijing, China Sponsors: NSF, DARPA

When is Rain Visible? Rain [Garg and Nayar, ICCV 05] Visibility of Rain Depends Strongly on Camera Parameters ( Exposure time, Aperture size, Focus distance )

Defining the Visibility of Rain Pixel Intensity Intensity Time (Frames) Visibility: Variance due to a Volume of Rain

Intensity of a Rain Streak background Background zmzm Rz m Fog like Rz m zmzm Noise Level Distance Change in Intensity Dependence on Defocus Dependence on Exposure Time Rain Region [Garg and Nayar, ICCV 05] Streak’s width Radius of blur circle Dependence on Distance

Visibility and Volume of Rain Background Number of drops Intensity Change Streak Width Variance Layer of Rain: Volume of Rain: Rain Properties Camera Properties Scene Properties [Garg and Nayar, ICCV 05]

Experimental Verification of Rain Visibility Standard Deviation Standard deviation F-number (N) Exposure time Visibility Distance of Focus Plane z 0 (m) (ms) Visibility Theoretical Visibility Measured Visibility (with error bars) Camera Parametes F-num (Aperture) Exposure Time Focus Distance zoom

Scene Near Depth Exposure F-number Motion Distance Range (m) Time (ms) slow close large small far large 33 6 small 33 2 fast close large X X small X X far large 8 6 small Optimal Settings for Rain Removal Slow: Motion< 15 pixels/secClose: z < Rain Visible RegionSmall: D z < Rain Visible Region

Scene with Fast Motion Default camera settings F-number =12 Exposure time = 8 ms Focus plane = 100 m

Scene with Fast Motion F-number decreased from F-number = 2.4 Exposure time = 8 ms Focus plane = 100 m

Comparison: Scene with Fast Motion Default camera settingsF-number :

Scene with Large Depth Default camera settings F-number =14 Exposure time = 16 ms Focus plane=10 m

Scene with Large Depth Exposure time Increased from ms F-number =14 Exposure time = 66 ms Focus plane=10 m

Comparison: Scene with Large Depth Default camera settingsExposure time: 16 66ms

Panning Video in Rain Default camera settings Exposure time =16 ms F-num=14 Optimal camera settings Exposure time =33 ms F-num=6

Implications for Vision Algorithms Default camera settings Exposure time =16 ms F-num=14 Optimal camera settings Exposure time =33 ms F-num=6 [KLT feature tracker code from Stan Birchfield]

Enhancing the Visibility of Rain: Rain Gauge Moderate Rain VideoHeavy Rain Video Type of RainfallCamera parameters (f, N, z0, T) Measured rain rate mm/hr Reported rain rate* mm/hr (a) Light(7095, 3.8, 3, 16) (b) Moderate(5148, 4.4, 3, 4) (c) Heavy(3300, 1.8, 3, 4)

Summary Contributions Future Work Camera Based Rain Gauge Analysis of Visibility of Rain Camera Parameters for Removal of Rain Limitations A Camera-Lens system that automatically detects rain in videos and sets optimal camera parameters for reducing rain during image acquisition Not as effective in very heavy rain and for scenes with close by and fast-moving objects