1. Omnidirectional Vision
Spring 2006
This course is a basic introduction to parts of the field of computer vision. This version of the course covers topics in 'early' or 'low' level vision and parts of 'intermediate' level vision. It assumes no background in computer vision, a minimal background in Artificial Intelligence and only basic concepts in calculus, linear algebra, and probability theory.
Lecture 3 - Part 2
Omnidirectional Cameras
Zhang Aiwu

2. Lecture Outline Applications
Robot navigation, Surveillance, Smart rooms
Video-conferencing/ Tele-presence
Multimedia/Visualization
GIS
Page of Omnidirectional Vision (Many universities and companies….)
Design Requirements
360 degree FOV, or semi-sphere or full sphere in one snapshot
Single effective viewpoint
Image Resolutions – one or more cameras?
Image Sharpness – optics as well as geometry
Several Important Designs
Catadioptric imaging : mirror (reflection) + lens ( refraction)
Mirrors: Planar, Conic, Spherical, Hyperboloidal, Ellipsoidal, Paraboloidal
Systematic design ( S. Nayar’s group)
Calibrations
Harder or simpler?

3. Which one?
Which one?
From the Page of Omnidirectional Vision

4. (Poly-)Dioptric solutions
One to two fish-eye cameras or many synchornized cameras
Homebrewed polydioptric cameras are cheaper, but require calibrating and synchronizing; commercial designs tend to be expensive

5. Catadioptric solutions
Usually single camera combined with convex mirror
Pros: - Single image
Cons: - Blindspot - Low resolution

6. Sensor Design Catadioptric imaging :
mirror (reflection) + lens ( refraction)
Theory of Catadioptric Image Formation ( S. Nayar’s group)
"A Theory of Single-Viewpoint Catadioptric Image Formation" , Simon Baker and Shree K. Nayar ,International Journal of Computer Vision, 1999.
Mirrors
Planar
Conic, Spherical
Hyperboloidal, Ellipsoidal
Paraboloidal
Cameras (Lens)
Perspective (pinhole) or orthogonal (tele-centric lens) projection
One or more?
Implementations
Compactness - size, support, and installation
Optics – Image sharpness, reflection, etc.

7. Planar Mirror
Panoramic camera system using a pyramid with four (or more) planar mirrors and four (or more) cameras (Nalwa96) has a single effective viewpoint
A very important concept in perspective geometry for computer vision and computer graphics
Mirror pyramid
6 cameras
4 camera design and 6 camera prototype:
FullView - Lucent Technology

8. Planar Mirror
Panoramic camera system using a pyramid with four (or more) planar mirrors and four (or more) cameras (Nalwa96) has a single effective viewpoint
A very important concept in perspective geometry for computer vision and computer graphics
Geometry of 4 camera approach: four separate cameras in 4 viewpoints can generate images with a single effective viewpoint

9. Planar Mirror Approach
A single effective viewpoint
More than one cameras
High image resolution
A very important concept in perspective geometry for computer vision and computer graphics

10. Planar Mirror Approach
A single effective viewpoint
More than one cameras
High image resolution
A very important concept in perspective geometry for computer vision and computer graphics

11. Conic Mirror Viewpoints on a circle
semispherical view except occlusion
Perspective projection in each direction
Robot Navigation (Yagi90, Zhu96/98)
viewpoint
pinhole
A very important concept in perspective geometry for computer vision and computer graphics

12. Viewpoints on a spherical-like surface
Spherical Mirror
Viewpoints on a spherical-like surface
Easy to construct (Hong91 -UMass )
Intersection of incoming rays are along this line
Locus of
viewpoints
A very important concept in perspective geometry for computer vision and computer graphics

13. Hyperboloidal Mirror Single Viewpoint
if the pinhole of the real camera and the virtual viewpoint are located at the two loci of the hyperboloid
Semi-spherical view except the self occlusion
pinhole P1
viewpoint P2
Rotation of the hyperbolic curve generates a hyperboloid
A very important concept in perspective geometry for computer vision and computer graphics

14. Hyperboloidal Mirror http://www.accowle.com/english/
ACCOWLE Co., LTD, A Spin-off at Kyoto University
Spherical Mirror
Hyperbolic Mirror
A very important concept in perspective geometry for computer vision and computer graphics
Image: High res. in the top

15. Ellipsoidal Mirror Single Viewpoint
if the pinhole of the real camera and the virtual viewpoint are located at the two loci of the ellipsoid
Semi-spherical view except the self occlusion
pinhole
viewpoint P1 P2
A very important concept in perspective geometry for computer vision and computer graphics

16. Panoramic Annular Lens
- geometric mathematical model
for image transform & calibration
p p1
pinhole P1 P B O C
Ellipsoidal mirror
Hyperboloidal mirror
panoramic annular lens (PAL)
- invented by Pal Greguss
* 40 mm in diameter, C-mount
* view: H: 360, V: -15 ~ +20
* single view point (O)

17. Panoramic Annular Lens
panoramic annular lens (PAL)
- invented by P. Greguss
* 40 mm in diameter, C-mount
* view: H: 360, V: -15 ~ +20
single view point (O)
C-Mount to CCD Cameras
Image: High res. In the bottom

18. Cylindrical panoramic un-warping
Two Steps:
(1). Center determination
(2) Distortion rectification
2-order polynomial approximation

19. Tele-lens - orthographic projection is used
Paraboloidal Mirror
Semi-spherical view except the self occlusion
Single Viewpoint at the locus of the paraboloid, if
Tele-lens - orthographic projection is used
Mapping between image, mirror and the world invariant to translation of the mirror. This greatly simplifies calibration and the computation of perspective images from paraboloidal images
A very important concept in perspective geometry for computer vision and computer graphics P1
viewpoint
tele-lens P2

20. Paraboloidal Mirror Remote Reality – A Spin-off at Columbia University
A very important concept in perspective geometry for computer vision and computer graphics
Camcorder
Web Camera
Back to Back : Full Spherical View

21. Paraboloidal Mirror Remote Reality – A Spin-off at Columbia University
A very important concept in perspective geometry for computer vision and computer graphics

22. Confused? Confused? Confused? Confused? Confused?
Q: What kind of sensor should one use?
A: Depends on your application.
1. If you are primarily concerned with:
– resolution – surveillance (coverage)
and can afford the bandwidth & expense, you might stick with polydioptric solutions
2. If you are concerned with
– bandwidth –servoing, SFM
investigate catadioptric or single wide FOV dioptric solutions

23. Application

24. Application

25. Application

26. Application

27. Application

28. 用GPS和全景相机 日本开发出绘城市地图新技术

29. Image Properties of Paraboloid System
(Assuming aspect ratio = 1)
The Image of a Line
is a circular arc if the line is not parallel to the optical axis
Is projected on a (radial) line otherwise
Dual Vanishing Points
There are two VPs for each set of parallel lines, which are the intersections of the corresponding circles
Collinear Centers
The center of the circles for a set of parallel lines are collinear
Vanishing Circle
The vanishing points of lines with coplanar directions* lie on a circle ( all the lines parallel to a common plane)

30. Image Properties of Paraboloid System
(with aspect ratio)
The Image Center
Is on the (“vanishing”) line connecting the dual vanishing points of each set of parallel lines
Can be determined by two sets of parallel lines
Projection of a Line with unknown aspect ratio
Is an elliptical arc in the general case
The Aspect Ratio
Is determined by the ratio of the lone-short axes of the ellipse corresponding to a line
Intrinsic Calibration
Estimate aspect ratio by the ratio of ellipse
Estimate the image center by the intersection of vanishing lines of two sets of parallel lines in 3-D space

31. Calibration of Paraboloid System
The Image Center
Is on the (“vanishing”) line connecting the dual vanishing points of each set of parallel lines
Can be determined by two sets of parallel lines

32. Calibration of Paraboloid System
The Image Center
Yellow “vanishing” line of horizontal set of parallel lines
Pink “vanishing” line of vertical set of parallel lines
The Vanishing Circle (Red dotted)
The vanishing points of lines with coplanar directions ( on a plane in this example)
Projected to the plane of the calibration pattern

33. Next
Turn in your projects and schedule meetings with me
END