CSCE 643 Computer Vision: Thinking in Frequency

Slides:



Advertisements
Similar presentations
Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean Hall 5409 T-R 10:30am – 11:50am.
Advertisements

Digital Image Processing
Linear Filtering – Part II Selim Aksoy Department of Computer Engineering Bilkent University
Reminder Fourier Basis: t  [0,1] nZnZ Fourier Series: Fourier Coefficient:
Image Filtering, Part 2: Resampling Today’s readings Forsyth & Ponce, chapters Forsyth & Ponce –
Convolution, Edge Detection, Sampling : Computational Photography Alexei Efros, CMU, Fall 2006 Some slides from Steve Seitz.
CSCE 641 Computer Graphics: Image Sampling and Reconstruction Jinxiang Chai.
Edges and Scale Today’s reading Cipolla & Gee on edge detection (available online)Cipolla & Gee on edge detection Szeliski – From Sandlot ScienceSandlot.
General Functions A non-periodic function can be represented as a sum of sin’s and cos’s of (possibly) all frequencies: F(  ) is the spectrum of the function.
CSCE 641 Computer Graphics: Image Sampling and Reconstruction Jinxiang Chai.
Image Sampling Moire patterns
Advanced Computer Graphics (Spring 2006) COMS 4162, Lecture 3: Sampling and Reconstruction Ravi Ramamoorthi
Lecture 3: Edge detection, continued
CSCE 641 Computer Graphics: Fourier Transform Jinxiang Chai.
Lecture 3: Image Resampling CS4670: Computer Vision Noah Snavely Nearest-neighbor interpolation Input image 3x upsample hq3x interpolation (ZSNES)
CPSC 641 Computer Graphics: Fourier Transform Jinxiang Chai.
Advanced Computer Graphics (Spring 2005) COMS 4162, Lecture 3: Sampling and Reconstruction Ravi Ramamoorthi
The Frequency Domain : Computational Photography Alexei Efros, CMU, Fall 2008 Somewhere in Cinque Terre, May 2005 Many slides borrowed from Steve.
Computational Photography: Fourier Transform Jinxiang Chai.
Image Sampling Moire patterns -
Fourier Analysis : Rendering and Image Processing Alexei Efros.
02/12/02 (c) 2002 University of Wisconsin, CS 559 Filters A filter is something that attenuates or enhances particular frequencies Easiest to visualize.
Image Sampling CSE 455 Ali Farhadi Many slides from Steve Seitz and Larry Zitnick.
Advanced Computer Graphics CSE 190 [Spring 2015], Lecture 3 Ravi Ramamoorthi
Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm.
Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am.
Topic 7 - Fourier Transforms DIGITAL IMAGE PROCESSING Course 3624 Department of Physics and Astronomy Professor Bob Warwick.
CSC589 Introduction to Computer Vision Lecture 7 Thinking in Frequency Bei Xiao.
Image Resampling CS4670: Computer Vision Noah Snavely.
Image Resampling CS4670: Computer Vision Noah Snavely.
Lecture 3: Edge detection CS4670/5670: Computer Vision Kavita Bala From Sandlot ScienceSandlot Science.
09/19/2002 (C) University of Wisconsin 2002, CS 559 Last Time Color Quantization Dithering.
Mestrado em Ciência de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informáticos VC 14/15 – TP6 Frequency Space Miguel Tavares Coimbra.
Lecture 7: Sampling Review of 2D Fourier Theory We view f(x,y) as a linear combination of complex exponentials that represent plane waves. F(u,v) describes.
Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm.
Recap of Monday linear Filtering convolution differential filters filter types boundary conditions.
Image as a linear combination of basis images
02/05/2002 (C) University of Wisconsin 2002, CS 559 Last Time Color Quantization Mach Banding –Humans exaggerate sharp boundaries, but not fuzzy ones.
Lecture 5: Fourier and Pyramids
Computer Graphics & Image Processing Chapter # 4 Image Enhancement in Frequency Domain 2/26/20161.
2D Fourier Transform.
Projects Project 1a due this Friday Project 1b will go out on Friday to be done in pairs start looking for a partner now.
Last Lecture photomatix.com. Today Image Processing: from basic concepts to latest techniques Filtering Edge detection Re-sampling and aliasing Image.
Frequency domain analysis and Fourier Transform
The Frequency Domain Digital Image Processing – Chapter 8.
Jean Baptiste Joseph Fourier
Advanced Computer Graphics
The Frequency Domain, without tears
Image Resampling & Interpolation
… Sampling … … Filtering … … Reconstruction …
Linear Filtering – Part II
Many slides from Steve Seitz and Larry Zitnick
Lecture 1.26 Spectral analysis of periodic and non-periodic signals.
Image Sampling Moire patterns
(C) 2002 University of Wisconsin, CS 559
Miguel Tavares Coimbra
Frequency domain analysis and Fourier Transform
General Functions A non-periodic function can be represented as a sum of sin’s and cos’s of (possibly) all frequencies: F() is the spectrum of the function.
Fourier Transform.
ENG4BF3 Medical Image Processing
Image Sampling Moire patterns
CSCE 643 Computer Vision: Image Sampling and Filtering
Samuel Cheng Slide credits: Noah Snavely
Image Processing Today’s readings For Monday
Filtering Part 2: Image Sampling
Image Sampling Moire patterns
Instructor: S. Narasimhan
Image Resampling & Interpolation
Resampling.
Samuel Cheng Slide credits: Noah Snavely
Presentation transcript:

CSCE 643 Computer Vision: Thinking in Frequency Jinxiang Chai

Image Scaling This image is too big to fit on the screen. How can we reduce it? How to generate a half- sized version?

Image Sub-sampling 1/8 1/4 Throw away every other row and column to create a 1/2 size image - called image sub-sampling

Image Sub-sampling 1/2 1/4 (2x zoom) 1/8 (4x zoom) Why does this look so crufty?

Difference between Lines

Even Worse for Synthetic Images

Really Bad in Video Click here

Aliasing occurs when your sampling rate is not high enough to capture the amount of detail in your image Can give you the wrong signal/image—an alias Where can it happen in computer graphics and vision? During image synthesis: sampling continuous signal into discrete signal e.g., ray tracing, line drawing, function plotting, etc. During image processing: resampling discrete signal at a different rate e.g. Image warping, zooming in, zooming out, etc. To do sampling right, need to understand the structure of your signal/image– signal processing

Signal Processing Analysis, interpretation, and manipulation of signals - images and videos - sampling and reconstruction of the signals. - minimal sampling rate for avoiding aliasing artifacts - how to use filtering to remove the aliasing artifacts?

Periodic Functions A periodic function is a function defined in an interval that repeats itself outside the interval

Jean Baptiste Fourier (1768-1830) had crazy idea (1807): Any periodic function can be rewritten as a weighted sum of sines and cosines of different frequencies. But it’s true! called Fourier Series

A Sum of Sine Waves Our building block: Add enough of them to get any signal f(x) you want!

A Sum of Sine Waves Our building block: Add enough of them to get any signal f(x) you want!

A Sum of Sine Waves Our building block: Add enough of them to get any signal f(x) you want!

A Sum of Sine Waves Our building block: Add enough of them to get any signal f(x) you want!

A Sum of Sine Waves Our building block: Add enough of them to get any signal f(x) you want! How many degrees of freedom? What does each control? Which one encodes the coarse vs. fine structure of the signal?

How about Non-peoriodic Function? A non-periodic function can also be represented as a sum of sin’s and cos’s But we must use all frequencies, not just multiples of the period The sum is replaced by an integral.

Fourier Transform A function f(x) can be represented as a sum of phase-shifted sine waves

Fourier Transform A function f(x) can be represented as a sum of phase-shifted sine waves How to compute F(u)?

Fourier Transform A function f(x) can be represented as a sum of phase-shifted sine waves How to compute F(u)?

Fourier Transform A function f(x) can be represented as a sum of phase-shifted sine waves How to compute F(u)? Amplitude: Phase angle:

Inverse Fourier Transform A function f(x) can be represented as a sum of phase-shifted sine waves How to compute F(u)? Inverse Fourier Transform Fourier Transform Amplitude: Phase angle:

Inverse Fourier Transform A function f(x) can be represented as a sum of phase-shifted sine waves How to compute F(u)? Inverse Fourier Transform Fourier Transform Dual property for Fourier transform and its inverse transform Amplitude: Phase angle:

Fourier Transform Magnitude against frequency: f(x) |F(u)| How much of the sine wave with the frequency u appear in the original signal f(x)?

Fourier Transform Magnitude against frequency: f(x) |F(u)| ? 5 How much of the sine wave with the frequency u appear in the original signal f(x)?

Fourier Transform Magnitude against frequency: f(x) |F(u)| 5 How much of the sine wave with the frequency u appear in the original signal f(x)?

Fourier Transform f(x) |F(u)| f(x) |F(u)|

Fourier Transform f(x) |F(u)| f(x) |F(u)|

Fourier Transform f(x) |F(u)| f(x) |F(u)|

Fourier Transform f(x) |F(u)| f(x) |F(u)|

Box Function and Its Transform f(x) x

Box Function and Its Transform f(x) x

Box Function and Its Transform f(x) x |F(u)| u If f(x) is bounded, F(u) is unbounded

Another Example If the fourier transform of a function f(x) is F(u), what is the fourier transform of f(-x)?

Another Example If the fourier transform of a function f(x) is F(u), what is the fourier transform of f(-x)?

Dirac Delta and its Transform f(x) x

Dirac Delta and its Transform f(x) x |F(u)| 1 u Fourier transform and inverse Fourier transform are qualitatively the same, so knowing one direction gives you the other

Cosine and Its Transform  -1 1 If f(x) is even, so is F(u)

Sine and Its Transform  -1 1 - If f(x) is odd, so is F(u)

Gaussian and Its Transform If f(x) is gaussian, F(u) is also guassian.

Gaussian and Its Transform If f(x) is gaussian, F(u) is also guassian. what’s the relationship of their variances?

Gaussian and Its Transform If f(x) is gaussian, F(u) is also guassian. what’s the relationship of their variances?

Properties Linearity:

Properties Linearity: Time-shift:

Properties Linearity: Time-shift:

Properties Linearity: Time-shift:

Properties Linearity: Time shift: Derivative: Integration: Convolution:

1D Signal Filtering

2D Image Filtering 49

2D Image Filtering 50

Signal Filtering A filter is something that attenuates or enhances particular frequencies 51

Spectrum of the filtered function Signal Filtering A filter is something that attenuates or enhances particular frequencies Easiest to visualize in the frequency domain, where filtering is defined as multiplication: Spectrum of the filtered function Spectrum of the filter Spectrum of the function 52

Filtering F G H Low-pass  = High-pass  = Band-pass  =

Filtering Identify filtered images from low-pass filter, high-pass filter, and band-pass filter?

Filtering in Frequency Domain H Low-pass  = High-pass  = Band-pass  = So what happens in the spatial domain? Convolution 55

Convolution Compute the integral of the product between f and a reversed and translated version of g represents the amount of overlap between f and a reversed and translated version of g.

Convolution Compute the integral of the product between f and a reversed and translated version of g represents the amount of overlap between f and a reversed and translated version of g.

Reversed and translated function g Convolution Compute the integral of the product between f and a reversed and translated version of g represents the amount of overlap between f and a reversed and translated version of g. Reversed and translated function g

Reversed and translated function g Convolution Compute the integral of the product between f and a reversed and translated version of g represents the amount of overlap/similarity between f and a reversed and translated version of g. Reversed and translated function g

Convolution Compute the integral of the product between f and a reversed and translated version of g t=2.5

Convolution Compute the integral of the product between f and a reversed and translated version of g t=4.6

Convolution Compute the integral of the product between f and a reversed and translated version of g t=4.6 For discrete signals such as images, this is just a weighted sum of all function values in the window - weights and window sizes are filter dependent 62

Convolution Theory

Convolution Theory

Convolution Theory

Convolution Theory

Convolution Theory

Qualitative Property The spectrum of a functions tells us the relative amounts of high and low frequencies Sharp edges give high frequencies Smooth variations give low frequencies A function is band-limited if its spectrum has no frequencies above a maximum limit sine, cosine are bandlimited Box, Gaussian, etc are not

Summary Convert a function from the spatial domain to the frequency domain and vice versa. Interpret a function in either domain, e.g., filtering and correlation. Build your intuition about functions and their spectra.