1. ASU MAT 591 Image Processing Science and Robotic Vision Rod Pickens Principal Research Engineer Lockheed Martin, Incorporated

2. Signals and Processing
Analog and discrete signals
Dimensionality of signals
1-D signals
Sounds (temporal), echocardiogram, seismic signal
2-D signals (this presentation)
Images (spatial)
3-D signals
Video sequences of images (spatial and temporal)
Signal processing
Synthesize and analyze signals
Filter signals using low-pass, band-pass, and high-pass filter
Modify signals such as warp, delay, stretch, rotate, shrink, …
Restore and enhance signals
Recognize patterns and detect signals

3. Signal Processing: Now
Animal
Robotic
Touch
Touch
Vision
Vision
Taste
Taste
Hearing
Hearing
Smell
Smell

4. The Processing Analogy

5. Analysis and Synthesis of Light
Fourier Synthesis
White Light In
Fourier Analysis
White Light Out
Inverse Functions

6. Fourier Transforms are Inverse Functions

7. Inverse Functions Derivative Inv Fourier Trans Inv Radon Trans
Warp Correction
Integral
Fourier Transform
Radon Transform
Warp Data

8. Filtering White Light In Red Light Out
Filtering removes all but red colors
Red Light Out

9. Television Television Stations 3, 5, 6, 13, 15, … Channel 6 Television
Filtering removes all but Channel 6
Channel 6

10. Television Television Stations 3, 5, 6, 13, 15, … Channel 15
Filtering removes all but Channel 15
Channel 15

11. Radio Radio Stations Radio Stations 91.5, 96.9, 100.7 Station 100.7
Filtering removes all but Station 100.7

12. Radio Radio Stations Radio Stations 91.5, 96.9, 100.7 Station 96.9
Filtering removes all but Station 96.9

13. Scene of a Room: walls, books, desks, chairs, windows,…
Vision
Scene of a Room: walls, books, desks, chairs, windows,…
Robot vision
Book
Filtering removes all but a book

14. Scene of a Room: walls, books, desks, chairs, windows,…
Vision
Scene of a Room: walls, books, desks, chairs, windows,…
Scene of a Room
Robot vision
Table
Filtering removes all but a table

15. Graphics to build a scene
Synthesis
Descriptor of scene is D(w)
All Room Contents
Scene of a Room

16. Filter that eliminates less important data.
Data compression
Signal
Filter that eliminates less important data.
Approximation
of Signal

17. Filter that eliminates less important data.
Data compression goal
Signal
Approximation
of Signal
Filter that eliminates less important data.

18. An Example of a Processing Architecture

19. The Example Architecture
Data
Format
Format
Correct
Errors
Communications
Correct
Errors
Preprocess
Preprocess
Normalize
Remove Noise
Remove Distortions
Restore
Remove Sensor Effects
Restore
Analyze
Decompose Signals
Analyze
Recognize
Label Signals
Recognize
Descriptions
Will Discuss in more detail!

20. Preprocess Data Format Correct Errors Preprocess Preprocess Restore
Normalize
Remove Noise
Remove Distortions
Analyze
Recognize
Descriptions

21. Fourier Based Noise Filtering
Noisy Input Image
Fourier Analysis
Mostly Noise so is Zeroed
Clearer Output Image
Mostly Signal
Fourier Synthesis
and
Filter the Noise
Fourier Transform
From Jason Plumb at

22. Filtering and Enhancing Data
Math to follow
From Mathworks homepage at

23. Filtering: Analysis
Image
Analysis

24. Filtering: Removing Noise
Image
Filtering: removes noise

25. Filtering: Synthesis
Image
Synthesis
Enhanced

26. Filtering
Enhanced
Filtering: removes noise
Image
Synthesis
Analysis

27. Enhancing the Data: Linear map
I2 = m* I1
Enhance (stretch)
Using Linear Mapping
I=Intensity I1
p(I1)
Input Image
Intensity Histogram I2
p(I2)
Output Image
Intensity Histogram
(more contrast)

28. Warping data Suppose we have unwanted camera motion.
From Mathworks homepage at

29. We can correct motion errors if we know motion model.
Warping data
We can correct motion errors if we know motion model.
From Mathworks homepage at

30. Warping data
From Mathworks homepage at

31. Warping Correction is an Inverse Function

32. Linear Algebra to Flip x1 y1 x2 y2

33. Linear Algebra to Flip x1 y1 x1 x2
x2=- x1 y1 y2
y2=y1 x2 y2

34. Linear Algebra to Flip y1
I(x1,y1) y2
y2=y1 x1 y1 x2 x2 y2 x1
x2=- x1

35. Linear Algebra to Flip y1 I(x1,y1) y2 y2=y1 x1 y1 x2 y2 x1 x2=- x1 x2

36. Linear Algebra to Flip y1 I(x1,y1) y2 y2=y1 x1 y1 x2 y2 y2 y2 y2 x1
x2=- x1 x2 x2 x2 x2
I(x2,y2)=I(f(x1),g(y1))

37. Linear Algebra to Flip y1 y1
y1=y2 x1 y2 x1 y2 x2
x1=- x2 x2
I(x2,y2)

38. Linear Algebra to Flip y1 I(f-1(x2), g-1(y2)) y1 y1=y2 x1 y2 x1 y2 x2
x1=- x2 x2
I(x2,y2)

39. Linear Algebra to Flip y1 y2 y2=y1 x1 y1 x2 y2 x1 x2=- x1 x2 I (x2,y2)
I (x1,y1)=I(f-1(x2), g-1(y2)) y2
y2=y1 x1 y1 x2 y2 x1
x2=- x1 x2
I (x2,y2)

40. Linear Algebra to Flip and Shrinky1 x1 y2 x2

41. Linear Algebra to Flip and Shrinky1 y2
y2 = -0.5 * y1 x1 y1 x2 y2
x2 = 0.5 * x1 x1 x2

42. Correcting warped data (camera motion)
If we can determine f(), g(), f-1(), and g-1(), then we can correct camera motion!
From Mathworks homepage at

43. Restoration Data Format Correct Errors Preprocess Restore Restore
Remove Sensor Effects
Analyze
Recognize
Descriptions

44. Restoring data for smear, optics,…
From Mathworks homepage at
Smear and optics can be viewed as filters that can degrade an image!
Uses
Linear Systems
Theory
Next

45. Restoring data for smear, optics,…
From Mathworks homepage at
Restoration
Uses
Linear Systems
Theory
Next

46. Restoration: Analysis
Image
Analysis

47. Filtering: Removing Smear
Image
Smr-1(wx,wy) is a filter that removes smear or restores the original object.

48. Filtering Smear inverted as a filter
Image
Object
Image Restored to best look like original Object

49. Restoring data for smear, optics,…
From Mathworks homepage at
Uses
Linear Systems
Theory
Image(wx,wy)
Next

50. Restoring data for smear, optics,…
From Mathworks homepage at
Smr(wx,wy)*Image(wx,wy)
Uses
Linear Systems
Theory
Image(wx,wy)
Next

51. Restoring data for smear, optics,…
From Mathworks homepage at
Smr(wx,wy)*Image(wx,wy)
Uses
Linear Systems
Theory
Image(wx,wy) *Smr-1(wx,wy)* Smr(wx,wy)
Image(wx,wy)= Image(wx,wy) *1(wx,wy )
Image(wx,wy)

52. Synthesis and Analysis
Data
Format
Correct
Errors
Preprocess
Restore
Synthesize
Analyze
Analyze
Recognize
Descriptions
Decompose / Compose Signals
- Transforms:
Fourier, SVD, Wavelets
- Statistical Analysis:
parametric and non-parametric

53. Fourier Transform Fourier Synthesis White Light In White Light Out
Fourier Analysis
White Light Out

54. Fourier Transform Magnitude Phase
From Wolfram homepage at
Magnitude
Phase

55. Radon Transform
From Mathworks homepage at

56. Wavelet Transform
From Wolfram homepage at

57. Many kinds of transforms
Common Transforms
Fourier
Discrete fourier
Cosine
Sine
Hough
Hadamard
Slant
Karhunen-Loeve
Fast KL
SVD
Sinusoidal
Many kinds of transforms

58. Statistics
From Mathworks homepage at

59. Recognition Data Format Correct Errors Preprocess Restore Analyze
Recognize
Recognize
Descriptions
Label Signals
- Signal Detection
- Pattern Recognition
- Artificial Intelligence

60. * Features are mathematical measurements
Pattern Recognition
Analysis
* Features are mathematical measurements
Class 2
(rose)
Class 1
(daisy)
Feature 1
Feature 1
Class 3
(sun flower)
Feature 2 *
Feature 2
Transforms: Fourier, Wavelet, …
Statistics: mean, st. dev, …
Shape: Fourier, Hough, Moments
Texture: Cooccurrence, Eigen Filters, …
Analysis Tools
Features
Feature 1: Hough measure
Feature 2: 3rd Eigen Filter
Classification
Bayesian
Neural nets
Nearest neighbors
Linear

61. Mathematical Decisions
Class 1 is z f2 z z z z z z o
How do we separate the classes? z z z o z z z o z o o z z o o o o o o f1 o o o o o o o
Class 2 is o

62. Mathematical Decisions
Class 1 is z f2 z z z z z z o z z z o z z
Linear decision z o z o o z z o o o o o o f1 o o o o o o o
Class 2 is o

63. Mathematical Decision
Class 1 is z f2 z z z z z z o z z z o z z
Linear decision z o z o o z z o o o o o o f1 o o o o o o o
Class 2 is o

64. Mathematical Decision
Class 1 is z f2 z z z z z z o z z z o z z
Quadratic decision z o z o o z z o o o o o o f1 o o o o o o o
Class 2 is o

65. Mathematical Decision
Class 1 is z f2 z z z z z z z z z z z z z z z f1

66. Mathematical Decisionf2 o o o o o o o o o o f1 o o o o o o o
Class 2 is o

67. Mathematical Decision
Class 1 is z f2 z z z z z z o z z z o z z z o z o o -1 3 z z o o o o o o f1 o o o o o o o
Class 2 is o

68. Isolate Object: Segmentation
Analysis
Synthesis
From Mathworks homepage at

69. Analyze Object: Features
- Length
- Width
- Contour
- Orientation
- Edges
Skeleton
- Texture Details
- Intensity
From Mathworks homepage at

70. Matched Filtering (registration)
Input Image or Iin(x,y)
From Mathworks homepage at

71. Matched Filtering (registration)
Input Image or Iin(x,y)
Exemplar (reference) or Iref(x,y)
From Mathworks homepage at

72. Matched Filtering (registration)
Input Image or Iin(x,y)
Exemplar (reference) or Iref(x,y)
error x x2
From Mathworks homepage at

73. Matched Filtering (registration)
Input Image or Iin(x,y)
Exemplar (reference) or Iref(x,y)
error x
Actually search form min of x,y simultaneously! x2
From Mathworks homepage at

74. Image Processing: Summary
Data
Format
Format
Correct
Errors
Communications
Correct
Errors
Preprocess
Preprocess
Normalize
Remove Noise
Remove Distortions
Restore
Remove Sensor Effects
Restore
Analyze
Decompose Signals
Analyze
Recognize
Label Signals
Recognize
Descriptions

75. References Fundamentals of Image Processing by Jain
Digital Image Analysis by Gonzalez and Wintz
Pattern Recognition by Fukunaga
Pattern Recognition Principles Tou and Gonzalez
Detection, Estimation, and Modulation Theory by Van Trees
Pattern Classification by Duda and Hart
Robot by Hans Moravec (graphics from

76. Signal Processing: 50 years from now
Evolved
Robotic
Touch
Touch
Hmmm.
Vision
Vision
Vision
Taste
Taste
Hearing
Hearing
Smell
Smell

77. Signal Processing: 50 years from now
Evolved
Robotic
Touch
Touch
Wow!
Vision
Vision
Vision
Taste
Taste
Hearing
Hearing
Smell
Smell

78. Signal Processing: 50 years from now
Evolved
Robotic
Touch
Touch
I see, therefore,
am I? Hmmm.
Vision
Vision
Vision
Taste
Taste
Hearing
Hearing
Smell
Smell