Computer Vision – Sampling Hanyang University Jong-Il Park

Department of Computer Science and Engineering, Hanyang University Introduction Topics  Television Standards (NTSC, SECAM, PAL)  Multi-dimensional Sampling Theory  Practical Limitations in Sampling and Reconstruction  Image Re-Sampling

Department of Computer Science and Engineering, Hanyang University

Department of Computer Science and Engineering, Hanyang University Television Standards Frame  525 lines/frame (or 625 lines/frame)  frame rate : 30 frames/s (or 25 frames/s) Field  even field, odd field  lines/field (or lines/field)

Department of Computer Science and Engineering, Hanyang University NTSC NTSC  525 scan lines/frame, 30 frames/s  line frequency : 15,750 Hz ( = 30 x 525 Hz )  2:1 line interlacing  color video composite signal - (Y,I,Q)  Bandwidth : Y - 4.2MHz, I - 1.3MHz, Q - 0.5MHz  color sub-carrier : MHz ( = 30 x 525 x 455/2 Hz)  phase change of 180  : between lines, between frames  Korea, North America, Japan etc.  Never Twice Same Color!

Department of Computer Science and Engineering, Hanyang University Other Standards SECAM(Sequential Couleur a Memoire)  Idea  avoid the quadrature demodulation and corresponding chrominance shift due to phase detection errors in NTSC  France, Eastern Europe.  625 lines/frame, 25 frames/s with 2:1 line interlace.  color video composite signal - (Y,U,V)  color sub-carrier : 4.25 MHz (for U) and 4.41 MHz (for V)  Something Essentially Contradictory to American Method!

Department of Computer Science and Engineering, Hanyang University Other Standards (cont.) PAL (Phase Alternating Line)  Idea  changes by 180 degree between successive line in the same field  cross talk can be suppressed Germany, UK, South America 625 lines at 25frames/s with 2:1 line interlace. color video composite signal - (Y,U,V)  (Bandwidth) Y - 4.2MHz, U - 1.3MHz, V - 1.3MHz  Peace At Last!

Department of Computer Science and Engineering, Hanyang University Sampling Theory For One-Dimensional Signal

Department of Computer Science and Engineering, Hanyang University f s =1/  t Nyquist Sampling Rate fs > 2B f S s (f)  -2f s -fs-fs fsfs 2fs2fs 0 f -B-BB S g (f) f S gs (f)  -2f s -fs-fs fsfs 2fs2fs 3fs3fs 4fs4fs reconstruction filter B-B-B Sampling Theory (cont.) For One-Dimensional Signal (cont.)

Department of Computer Science and Engineering, Hanyang University Fourier Transform of a bandlimited function Its region of support Sampling Theory(cont.) For Two-Dimensional Signal  Band-limited Image

Department of Computer Science and Engineering, Hanyang University Sampling Theory (cont.) For Two-Dimensional Signal (cont.)  Structure  Orthogonal Structure (Rectangular Tesselation)  Field Quincunx Structure (Triangular Tesselation)

Department of Computer Science and Engineering, Hanyang University Sampling Theory(cont.) For Two-Dimensional Signal(cont.)  Structure(cont.)

Department of Computer Science and Engineering, Hanyang University

Department of Computer Science and Engineering, Hanyang University 2D sampling For Two-Dimensional Signal (cont.)  Orthogonal Structure (Rectangular Tesselation)  Sampling Function

Department of Computer Science and Engineering, Hanyang University 2D sampling - Spectrum For Two-Dimensional Signal (cont.)  Orthogonal Structure (cont.)  Spectrum of sampled signals x y yy xx u v vv uu u v vv uu (a) sampling function (b) spectrum of sampling function and signal (c) spectrum of sampled signal xx yy

Department of Computer Science and Engineering, Hanyang University For Two-Dimensional Signal(cont.)  Orthogonal Structure(cont.)  Reconstruction of the original image from its samples  Nyquist Sampling Rate(or Frequency) and Nyquist Interval 2D sampling - Reconstruction

Department of Computer Science and Engineering, Hanyang University Reconstruction Filter

Department of Computer Science and Engineering, Hanyang University Aliasing effect For Two-Dimensional Signal(cont.)  Orthogonal Structure(cont.)  Aliasing Effect

Department of Computer Science and Engineering, Hanyang University Zone Plate image (  = 1) Aliasing (  = 2) Eg. Aliasing For Two-Dimensional Signal (cont.)  Orthogonal Structure (cont.)  Aliasing Effect (cont.)

Department of Computer Science and Engineering, Hanyang University Eg. Aliasing Little aliasing due to an effective antialiasing filter Noticeable aliasing Examples

Department of Computer Science and Engineering, Hanyang University Practical limitations in sampling Practical Limitations  Real-world images are not band-limited.  aliasing errors  can be reduced by LPF before sampling  LPF attenuate higher spatial frequencies  Resolution loss  blurring  No ideal LPF at reconstruction stage.

Department of Computer Science and Engineering, Hanyang University Sampling aperture Finite aperture (finite duration pulse)

Department of Computer Science and Engineering, Hanyang University Sampling aperture - Spectrum Practical Limitations (cont.)  Sampling Aperture/ LPF operation

Department of Computer Science and Engineering, Hanyang University Reconstruction Reconstruction of a signal from its sample using interpolation

Department of Computer Science and Engineering, Hanyang University Linear interpolation linear interpolation

Department of Computer Science and Engineering, Hanyang University

Department of Computer Science and Engineering, Hanyang University Sampling Theory(cont.)

Department of Computer Science and Engineering, Hanyang University Geometrical Image Resampling Bilinear interpolation

Department of Computer Science and Engineering, Hanyang University Geometrical Image Resampling(Cont.) Bicubic interpolation

Department of Computer Science and Engineering, Hanyang University Convolution methods : integer zoom(ex. 2 : 1)  Zero Interleaving  Convolution  Peg : Pyramid : Bell :  Cubic B-spline : Resampling by Convolution

Department of Computer Science and Engineering, Hanyang University Original Zero interleaving Eg: Resampling by Convolution

Department of Computer Science and Engineering, Hanyang University Bell Cubic B-spline Eg: Resampling by Convolution(Cont.) peg Pyramid

Department of Computer Science and Engineering, Hanyang University Image Warping image filtering: change range of image g(x) = h(f(x)) image warping: change domain of image g(x) = f(h(x)) f x h f x f x h f x

Department of Computer Science and Engineering, Hanyang University Image Warping image filtering: change range of image g(x) = h(f(x)) image warping: change domain of image g(x) = f(h(x)) hh f f g g

Department of Computer Science and Engineering, Hanyang University Parametric (global) warping Examples of parametric warps: translation rotation aspect affine perspective cylindrical

Department of Computer Science and Engineering, Hanyang University 2D Coordinate Transformations translation:x’ = x + t x = (x,y) rotation:x’ = R x + t similarity:x’ = s R x + t affine:x’ = A x + t perspective:x’  H x x = (x,y,1) (x is a homogeneous coordinate)  These all form a nested group

Department of Computer Science and Engineering, Hanyang University Image Warping Given a coordinate transform x’ = h(x) and a source image f(x), how do we compute a transformed image g(x’) = f(h(x))? f(x)f(x)g(x’) xx’ h(x)h(x)

Department of Computer Science and Engineering, Hanyang University Forward Warping Send each pixel f(x) to its corresponding location x’ = h(x) in g(x’) f(x)f(x)g(x’) xx’ h(x)h(x) What if pixel lands “between” two pixels?

Department of Computer Science and Engineering, Hanyang University Forward Warping Send each pixel f(x) to its corresponding location x’ = h(x) in g(x’) f(x)f(x)g(x’) xx’ h(x)h(x) What if pixel lands “between” two pixels? Answer: add “contribution” to several pixels, normalize later (splatting)

Department of Computer Science and Engineering, Hanyang University Inverse Warping Get each pixel g(x’) from its corresponding location x = h -1 (x’) in f(x) f(x)f(x)g(x’) xx’ h -1 (x’) What if pixel comes from “between” two pixels?

Department of Computer Science and Engineering, Hanyang University Inverse Warping Get each pixel g(x’) from its corresponding location x = h -1 (x’) in f(x) What if pixel comes from “between” two pixels? Answer: resample color value from interpolated (prefiltered) source image f(x)f(x)g(x’) xx’

Department of Computer Science and Engineering, Hanyang University Interpolation Possible interpolation filters:  nearest neighbor  bilinear  bicubic (interpolating)  sinc / FIR Needed to prevent “jaggies” and “texture crawl”