1. Shape-from-Polarimetry: A New Tool for Studying the Air-Sea Interface Shape-from-Polarimetry: Howard Schultz, UMass Amherst, Dept of Computer Science Chris J. Zappa, Michael L. Banner, Russel Morison, Larry Pezzaniti Howard Schultz, UMass Amherst, Dept of Computer Science Chris J. Zappa, Michael L. Banner, Russel Morison, Larry Pezzaniti

2. Introduction

3. What is Polarimetry – Light has 3 basic qualities – Color, intensity and polarization – Humans do not see polarization Introduction

4. Linear Polarization http://www.enzim.hu/~szia/cddemo/edemo0.htm

5. Circular Polarization

6. A bundle of light rays is characterized by intensity, a frequency distribution (color), and a polarization distribution Polarization distribution is characterized by Stokes parameters S = (S0, S1, S2, S3) The change in polarization on scattering is described by Muller Calculus S OUT = M S IN Where M contains information about the shape and material properties of the scattering media The goal: Measure S OUT and S IN and infer the parameters of M Muller Calculus Amount of circular polarization Orientation and degree of linear polarization Intensity Incident Light Muller Matrix Scattered Light

7. Shape-from-Polarimetry (SFP) Use the change in polarization of reflected or refracted skylight to infer the 2D surface slope,, for every pixel in the imaging polarimeter’s field-of-view

8. Shape-from-Polarimetry S = S AW + S WA S AW = R AW  S SKY and S WA = T AW  S UP Kattawar, G. W., and C. N. Adams (1989), “Stokes vector calculations of the submarine light-field in an atmosphere-ocean with scattering according to a Rayleigh phase matrix - Effect of interface refractive-index on radiance and polarization,” Limnol. Oceanogr., 34(8),1453-1472.

9. Shape-from-Polarimetry For simplicity we incorporated 3 simplifying assumptions – Skylight is unpolarized S SKY = S SKY (1,0,0,0) good for overcast days – In deep, clear water upwelling light can be neglected S WA = (0,0,0,0). – The surface is smooth within the pixel field-of-view

10. Shape-from-Polarimetry Sensitivity =  (DOLP) /  θ

11. Experiments Conduct a feasibility study – Rented a linear imaging polarimeter – Laboratory experiment setup a small 1m x 1m wavetank Used unpolarized light Used wire gauge to simultaneously measure wave profile – Field experiment Collected data from a boat dock Overcast sky (unpolarized) Used a laser slope gauge

13. Looking at 90  to the waves Looking at 45  to the waves Looking at 0  to the waves

15. Slope in Degrees X-Component Y-Component

16. X-ComponentY-Component

18. Build an Imaging Polarimeter for Oceanographic Applications – Polaris Sensor Technologies – Funded by an ONR DURIP – Frame rate 60 Hz – Shutter speed as short as 10 μsec – Measure all Stokes parameters – Rugged and light weight – Deploy in the Radiance in a Dynamic Ocean (RaDyO) research initiative http://www.opl.ucsb.edu/radyo/

19. Motorized Stage 12mm travel 5mm/sec max speed Objective Assembly Polarizing beamsplitter assembly Camera 1 (fixed) Camera 2 Camera 3 Camera 4

23. Air-Sea Flux Package Imaging Polarimeter Scanning Altimeters and Visible Camera ~36° Deployed during the ONR experiment Radiance in a Dynamic Ocean (RaDyO)

24. Analysis & Conclusion A sample dataset from the Santa Barbara Channel experiment was analyzed Video 1 shows the x- and y-slope arrays for 1100 frames Video 2 shows the recovered surface (made by integrating the slopes) for the first 500 frames

25. Time series comparison

27. Convert slope arrays to a height array

28. Convert slope arrays to a height array (Integration)

29. Convert slope arrays to a height array Use the Fourier derivative theorem

30. Reconstructed Surface Video

31. Analysis & Conclusion The shape-from-polarimetry method works well for small waves in the 1mm to 10cm range. Need to improve the theory by removing the three simplifying assumptions – Skylight is unpolarized S SKY = S SKY (1,0,0,0) – Upwelling light can be neglected S WA = (0,0,0,0). – The surface is smooth within the pixel field-of-view Needs to have an independent estimate of lower frequency waves.

32. Seeing Through Waves Sub-surface to surface imaging Surface to sub-surface imaging

33. Optical Flattening

34. Remove the optic distortion caused by surface waves to make it appear as if the ocean surface was flat – Use the 2D surface slope field to find the refracted direction for each image pixel – Refraction provides sufficient information to compensate for surface wave distortion – Real-time processing

35. Image Formation Subsurface-to-surface Imaging Array Exposure Center Observation Rays Air Water

36. Image Formation surface-to-subsurface Imaging Array Exposure Center Air Water Imaging Array Exposure Center

37. Seeing Through Waves

38. 0 20 40 60 800 10 20 30 40 Seeing Through Waves

39. Optical Flattening Remove the optic distortion caused by surface waves to make it appear as if the ocean surface was flat – Use the 2D surface slope field to find the refracted direction for each image pixel – Refraction provides sufficient information to compensate for surface wave distortion – Real-time processing

40. Un-distortion A lens maps incidence angle θ to image position X Lens Imaging Array X θ

41. X θ Lens Imaging Array Un-distortion A lens maps incidence angle θ to image position X

42. X Lens Imaging Array Un-distortion A lens maps incidence angle θ to image position X

43. X θ Lens Imaging Array Un-distortion A lens maps incidence angle θ to image position X

44. X θ Lens Imaging Array Un-distortion A lens maps incidence angle θ to image position X

45. Distorted Image Point Image array Un-distortion Use the refraction angle to “straighten out” light rays Air Water

46. Un-distorted Image Point Image array Un-distortion Use the refraction angle to “straighten out” light rays Air Water

47. Real-time Un-Distortion The following steps are taken Real-time Capable – Collect Polarimetric Images ✔ – Convert to Stokes Parameters ✔ – Compute Slopes (Muller Calculus) ✔ – Refract Rays (Lookup Table) ✔ – Remap Rays to Correct Pixel ✔

48. Image Formation surface-to-subsurface Imaging Array Exposure Center Air Water Imaging Array Exposure Center

49. Detecting Submerged Objects “Lucky Imaging” Use refraction information to keep track of where each pixel (in each video frame) was looking in the water column Build up a unified view of the underwater environment over several video frames Save rays that refract toward the target area Reject rays that refract away from the target area

51. For more information contact Howard Schultz University of Massachusetts Department of Computer Science 140 Governors Drive Amherst, MA 01003 Phone: 413-545-3482 Email: hschultz@cs.umass.edu