Download presentation
Presentation is loading. Please wait.
Published byPhoebe Ford Modified over 9 years ago
1
A Tool for Signal Synthesis & Analysis Robert J. Marks II, Distinguished Professor of Electrical and Computer Engineering Baylor University RobertMarks@RobertMarks.org
2
Outline POCS: What is it? –Convex Sets –Projections –POCS Applications –The Papoulis-Gerchberg Algorithm –Neural Network Associative Memory –Resolution at sub-pixel levels –Radiation Oncology / Tomography –JPG / MPEG repair –Missing Sensors
3
POCS: What is a convex set? In a vector space, a set C is convex iff and, C
4
POCS: Example convex sets Convex Sets Sets Not Convex
5
POCS: Example convex sets in a Hilbert space Bounded Signals – a box then, if 0 1 x ( t )+( 1 - ) y ( t ) u. If x ( t ) u and y ( t ) u, x(t)x(t) t u
6
POCS: Example convex sets in a Hilbert space Identical Middles – a plane t c(t)c(t) x(t)x(t) t c(t)c(t) y(t)
7
POCS: Example convex sets in a Hilbert space Bandlimited Signals – a plane (subspace) B = bandwidth If x(t) is bandlimited and y(t) is bandlimited, then so is z(t) = x(t) + (1- ) y(t)
8
POCS: Example convex sets in a Hilbert space Fixed Area Signals – a plane x(t)x(t) t A y(t) t A
9
POCS: Example convex sets in a Hilbert space Identical Tomographic Projections – a plane y p(y) p(y) y x
10
POCS: Example convex sets in a Hilbert space Identical Tomographic Projections – a plane y p(y) p(y) y x
11
POCS: Example convex sets in a Hilbert space Bounded Energy Signals – a ball 2E2E
12
POCS: Example convex sets in a Hilbert space Bounded Energy Signals – a ball 2E2E
13
POCS: What is a projection onto a convex set? For every (closed) convex set, C, and every vector, x, in a Hilbert space, there is a unique vector in C, closest to x. This vector, denoted P C x, is the projection of x onto C: C x P C x y P C y zP C z=
14
Example Projections Bounded Signals – a box y(t) t u P C y(t) y(t)y(t)
15
Example Projections Identical Middles – a plane y(t)y(t) c(t)c(t) t P C y(t) t y(t) C P C y(t)
16
y(t) y(t) Example Projections Bandlimited Signals – a plane (subspace) y(t)y(t) C P C y(t) Low Pass Filter: Bandwidth B PC y(t)PC y(t)
17
Example Projections Constant Area Signals – a plane (linear variety) x[1] x[2]x[1]+ x[2]=A A A Continuous Discrete
18
Example Projections Constant Area Signals – a plane (linear variety) x[1] x[2] x[1]+ x[2]=A A A Same value added to each element. y(t)y(t) C P C y(t)
19
y p(y) p(y) y x Example Projections Identical Tomographic Projections – a plane g(x,y) P C g(x,y) C
20
Example Projections Bounded Energy Signals – a ball 2E2E P C y(t) y(t)y(t)
21
The intersection of convex sets is convex C1C1 C2C2 } C 1 C 2 C1C1 C2C2 C 1 C 2
22
Alternating POCS will converge to a point common to both sets C1C1 C2C2 = Fixed Point The Fixed Point is generally dependent on initialization
23
Lemma 1: Alternating POCS among N convex sets with a non-empty intersection will converge to a point common to all. C2C2 C3C3 C1C1
24
Lemma 2: Alternating POCS among 2 nonintersecting convex sets will converge to the point in each set closest to the other. C2C2 C1C1 Limit Cycle
25
Lemma 3: Alternating POCS among 3 or more convex sets with an empty intersection will converge to a limit cycle. C1C1 C3C3 C2C2 Different projection operations can produce different limit cycles. 1 2 3 Limit Cycle 3 2 1 Limit Cycle
26
Lemma 3: (cont) POCS with non-empty intersection. C1C1 C3C3 1 2 3 Limit Cycle 3 2 1 Limit Cycle C2C2
27
Simultaneous Weighted Projections C1C1 C3C3 C2C2
28
Outline POCS: What is it? –Convex Sets –Projections –POCS Diverse Applications –The Papoulis-Gerchberg Algorithm –Neural Network Associative Memory –Resolution at sub-pixel levels –Radiation Oncology / Tomography –JPG / MPEG repair –Missing Sensors
29
Application: The Papoulis-Gerchberg Algorithm Restoration of lost data in a band limited function: g(t) g(t) ? ? f(t) f(t) BL ? Convex Sets: (1) Identical Tails and (2) Bandlimited.
30
Application: The Papoulis-Gerchberg Algorithm Example
31
Outline POCS: What is it? –Convex Sets –Projections –POCS Applications –The Papoulis-Gerchberg Algorithm –Neural Network Associative Memory –Resolution at sub-pixel levels –Radiation Oncology / Tomography –JPG / MPEG repair –Missing Sensors
32
A Plane! Application: Neural Network Associative Memory Placing the Library in Hilbert Space
33
Application: Neural Network Associative Memory Associative Recall Identical Pixels
34
Application: Neural Network Associative Memory Associative Recall by POCS Identical PixelsColumn Space
35
Application: Neural Network Associative Memory Example 23 5 1 1019 4
36
Application: Neural Network Associative Memory Example 12 4 0 1019 3
37
A Library of Mathematicians What does the Average Mathematician Look Like?
38
Convergence As a Function of Percent of Known Image
39
Convergence As a Function of Library Size
40
Convergence As a Function of Noise Level
41
Combining Euler & Hermite Not Of This Library
42
Outline POCS: What is it? –Convex Sets –Projections –POCS Applications –The Papoulis-Gerchberg Algorithm –Neural Network Associative Memory –Resolution at sub-pixel levels –Radiation Oncology / Tomography –JPG / MPEG repair –Missing Sensors
43
Application: Combining Low Resolution Sub-Pixel Shifted Images http://www.stecf.org/newsletter/stecf-nl-22/adorf/adorf.html Hans-Martin Adorf http://www.stecf.org/newsletter/stecf-nl-22/adorf/adorf.html Problem: Multiple Images of Galazy Clusters with subpixel shifts. POCS Solution: 1.Sets of high resolution images giving rise to lower resolution images. 2.Positivity 3.Resolution Limit 2 4
44
Application: Subpixel Resolution Object is imaged with an aperture covering a large area. The average (or sum) of the object is measured as a single number over the aperture. The set of all images with this average value at this location is a convex set. The convex set is constant area.
45
Application: Subpixel Resolution Randomly chosen 8x8 pixel blocks. Over 6 Million projections.
46
Slow (7:41) Fast (3:00) Faster (1:30) Fastest (0:56)SlowFastFasterFastest Application: Subpixel Resolution Randomly chosen 8x8 pixel blocks. Over 6 Million projections.
47
Application: Subpixel Resolution Blocks of random dimension <33 chosen at random locations. 2.7 Million blocks. Slow (0.58) Fast (0:29) Faster (0:14)SlowFastFaster
48
Outline POCS: What is it? –Convex Sets –Projections –POCS Applications –The Papoulis-Gerchberg Algorithm –Neural Network Associative Memory –Resolution at sub-pixel levels –Radiation Oncology / Tomography –JPG / MPEG repair –Missing Sensors
49
Application: Tomography Same procedure as subpixel resolution.
52
Tomography Same procedure as subpixel resolution. 102k iterations in ~log time Slow (0.19) Fast (0:09 )SlowFast
53
Tomography Same procedure as subpixel resolution. 409k iterations in ~log time Slow Slow (0.43) Fast (0:21 )Fast
54
Application: Radiation Oncology Conventional Therapy Illustration
55
Application: Radiation Oncology Intensity Modulated Radiotherapy
56
Application: Radiation Oncology Convex sets for dosage optimization N = normal tissue T = tumor C = critical organ C N = Set of beam patterns giving dosage in N less than T N. C C = Set of beam patterns giving dosage in C less than T C <T N. C T = Set of beam patterns giving dosage in T between T Low and T hi (no cold spots or hot spots). C C = Set of nonnegative beam patterns.
57
Application: Radiation Oncology C T N Example POCS Solution
58
Outline POCS: What is it? –Convex Sets –Projections –POCS Applications –The Papoulis-Gerchberg Algorithm –Neural Network Associative Memory –Resolution at sub-pixel levels –Radiation Oncology / Tomography –JPG / MPEG repair –Missing Sensors
59
Application Block Loss Recovery Techniques for Image and Video Communications Transmission & Error During jpg or mpg transmission, some packets may be lost due to bit error, congestion of network, noise burst, or other reasons. Assumption: No Automatic Retransmission Request (ARQ) Missing 8x8 block of pixels
60
Application Projections based Block Recovery – Algorithm Two Steps Edge orientation detection POCS using Three Convex Sets
61
Application Recovery vectors are extracted to restore missing pixels. Two positions of recovery vectors are possible according to the edge orientation. Recovery vectors consist of known pixels(white color) and missing pixels(gray color). The number of recovery vectors, r k, is 2. Vertical line dominating area Horizontal line dominating area
62
Application Forming a Convex Constraint Using Surrounding Vectors Surrounding Vectors, s k, are extracted from surrounding area of a missing block by N x N window. Each vector has its own spatial and spectral characteristic. The number of surrounding vectors, s k, is 8N.
63
Application Projections based Block Recovery – Projection operator P 1 Convex Cone Hull Surrounding Blocks
64
Application Projection operator P 1 Convex Cone Hull P 1 r r
65
Application Projection operator P 2 Identical Middles Keep This Part Replace this part with the known image P1P1
66
Convex set C 3 acts as a convex constraint between missing pixels and adjacent known pixels, (f N-1 f N ). The rms difference between the columns is constrained to lie below a threshold. Application Projection operator P 3 Smoothness Constraint f N-1 f N
67
Application Simulation Results – Test Data and Error Peak Signal to Noise Ratio
68
Simulation Results – Lena, 8 x 8 block loss Original ImageTest Image
69
Simulation Results – Lena, 8 x 8 block loss Ancis, PSNR = 28.68 dBHemami, PSNR = 31.86 dB
70
Simulation Results – Lena, 8 x 8 block loss Ziad, PSNR = 31.57 dBPOCS, PSNR = 34.65 dB
71
Simulation Results – Lena, 8 x 8 block loss Ancis PSNR = 28.68 dB Hemami PSNR = 31.86 dB Ziad PSNR = 31.57 dB POCS PSNR = 34.65 dB
72
Simulation Results – Each Step Lena 8 x 8 block loss (a) (b) (c)
73
Simulation Results – Peppers, 8 x 8 block loss Original ImageTest Image
74
Simulation Results – Peppers, 8 x 8 block loss Ancis, PSNR = 27.92 dBHemami, PSNR = 31.83 dB
75
Simulation Results – Peppers, 8 x 8 block loss Ziad, PSNR = 32.76 dBPOCS, PSNR = 34.20 dB
76
Simulation Results – PSNR (8 x 8) LenaMasqrdPeppersBoatElaineCouple Ancis28.6825.4727.9226.3329.8428.24 Sun29.9927.2529.9727.3630.9528.45 Park31.2627.9131.7128.7732.9630.04 Hemami31.8627.6531.8329.3632.0730.31 Ziad31.5727.9432.7630.1131.9230.99 POCS34.6529.8734.2030.7834.6331.49
77
Simulation Results – Masquerade, 8 x one row block loss Original ImageTest Image
78
Simulation Results – Masquerade, 8 x one row block loss Hemami, PSNR = 23.10 dBPOCS, PSNR = 25.09 dB
79
Interpolation based Coding – Result 1 JPEG Coding PSNR = 32.27 dB Size = 0.30 BPP = 9,902 Byte w/ Removed Blocks Blocks : 447 / 4096 = 11% Size = 0.29 BPP I-based Coding PSNR = 32.35 dB Size = 0.29 BPP = 9,634 Byte
80
Interpolation based Coding – Result 2 JPEG Coding PSNR = 32.27 dB Size = 0.30 BPP = 9,902 Byte w/ Removed Blocks Blocks : 557 / 4096 = 14% Size = 0.27 BPP I-based Coding PSNR = 32.37 dB Size = 0.27 BPP = 9,570 Byte
81
Temporal Block Loss Recovery In video coding (e, g, MPEG), temporal recovery is more effective. t t- 1
82
Simulation Results – Flower Garden Original Sequence Test Sequence
83
Simulation Results – Flower Garden Zero Motion Vector, PSNR = 16.15 dB Average of Surrounding Motion Vectors, PSNR = 18.64 dB
84
Simulation Results – Flower Garden Motion Flow Interpolation (1999), PSNR = 19.29 dB Boundary Matching Algorithm (1993), PSNR = 19.83 dB
85
Simulation Results – Flower Garden Decoder Motion Vector Estimation (2000), PSNR = 19.21 dB POCS Based, PSNR = 20.71 dB
86
Simulation Results – Foreman Original SequenceTest Sequence Zero Motion Vector PSNR = 24.71 dB Average of Surrounding Motion Vectors PSNR = 26.22 dB
87
Simulation Results – Foreman Motion Flow Interpolation (1999) PSNR = 27.09 dB Boundary Matching Algorithm (1993), PSNR = 28.76 dB Decoder Motion Vector Estimation (2000), PSNR = 27.46 dB POCS Based PSNR = 29.82 dB
88
Simulation Results – Average PSNR GardenTennisFootballMobileForeman MV16.1522.4018.0617.4924.71 AV18.6421.9818.7219.0326.22 BMA19.8323.5519.4119.7528.76 DMVE19.8824.0419.6420.0228.77 MFI19.2922.7719.2919.6027.09 F-B BM19.2122.4919.0519.5927.46 Proposed20.7124.5220.3220.6629.82
89
Outline POCS: What is it? –Convex Sets –Projections –POCS Applications –The Papoulis-Gerchberg Algorithm –Neural Network Associative Memory –Resolution at sub-pixel levels –Radiation Oncology / Tomography –JPG / MPEG repair –Missing Sensors
90
Application Missing Sensors Idea: A collection of spatially distributed point sensors. Their readings are interrelated (e.g. temperature sensors in a room).
91
Application Missing Sensors A plurality of sensors fail. X X X X X Can the failed sensor readings be regained from those remaining without use of models? Applications: (1) Power Security Assessment (2) Engine Vibration Sensors
92
Application Missing Sensors Step 1: Learn the interrelationship among the sensors by training an auto-encoder neural network with historical data. The mapping of a properly trained neural network is a projection.
93
Application Missing Sensors Step 2: Impose the second convex constraint of known sensor values. If POCS, convergence is assured ignore ignoreignore g
94
Application Missing Sensors Example: Vibration Sensors on a Jet Engine x direction y direction z direction
95
Application Missing Sensors Four Sensors – One Missing known restored Original Sensor Data Reconstructed sensor data when sensor 4 fails.
96
Application Missing Sensors Four Sensors – Two Missing ail Figure 5: Restoration of two lost vibration sensors corresponding to 44 points of data. The real known missing
97
Application Missing Sensors Eight Sensors – Two Missing (Magnitude Only) ail Figure 5: Restoration of two lost vibration sensors corresponding to 44 points of data. The real Reconstructed sensor data when sensors seven and eight fail. missing Reconstruction Error known
98
Final Comments POCS: What is it? –Convex Sets –Projections –POCS Applications –The Papoulis-Gerchberg Algorithm –Neural Network Associative Memory –Resolution at sub-pixel levels –Radiation Oncology / Tomography –JPG / MPEG repair –Missing Sensors
100
Finis
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.