Accidental pinhole and pinspeck cameras: revealing the scene outside the picture A. Torralba and W. T. Freeman Proceedings of 25 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2012)

Outlines Pinhole camera Accidental pinhole image Anti-pinhole/pinspeck camera Accidental anti-pinhole image Applications of accidental images in scenes

Pinhole camera How does it work?

Accidental pinhole image Unnoticed or misinterpreted as shadows.

Accidental pinhole image Windows turns into a pinhole

Accidental Pinhole Camera Images are always blurry. To get sharp image, we need to make small aperture. It is unlikely to happen accidentally

Pinspeck Camera

Two image: 1) image with whole light rays ;2) image with an occluder; The difference of two image.

Accidental Anti-pinhole Camera The situation happens quite often, and is always ignored. There are two limitations in this phenomenon: – It needs a reference image. – Signal to noise ratio.

Accidental Anti-pinhole Camera Use two images or a video to extract referenced image; Use temporal integration to improve signal to noise ratio.

Example Two different “cameras”

Example Two different images

Example Two difference image and original view.

Applications of accidental images in scenes Seeing what is outside the room Seeing light source Seeing the shape of window Seeing the illumination map in an outdoor scene

Seeing what is outside the room Light comes from the window of a room A person walk around in the room.

Seeing light sources A hand as an occluder.

Seeing the shape of window The shape of the window modifies the statistics of the intensities seeing on the wall just as a blur kernel changes the statistics of a sharp image. This motivates using algorithm from image deblurring to infer the shape of window.

Seeing the shape of window

Seeing the illumination map in an outdoor scene