1. Compressive Coded Aperture Video Reconstruction
Roummel F. Marcia,
Rebecca M. Willett
European Signal Processing Conf. (EUSIPCO), August 2008

2. Outline Introduction Problem Formulation
Compressive sensing
Compressive coded aperture mask design
Video setting
Sparse Representation Algorithm
Simulation

3. Introduction
Many existing techniques for increasing video resolution are based on superresolution image reconstruction. These approaches, however, typically either require relatively large numbers of observed pixels or yield unsatisfactory edges and boundaries.
This paper describes a novel technique for increasing the resolution of digital video using a combination of coded aperture sensing and wavelet-based reconstruction algorithms.

4. Introduction
Coded aperture masks based on Toeplitz-structured matrices for compressive sensing can be used to reconstruct high resolution static images from low-resolution coded observations.
There is a tradeoff between the sparsity of the vector being reconstructed and computation time.
If each frame process independently, the problem size would be small and each iteration of a reconstruction method would require little computation time.
If a block of sequential frames were processed simultaneously, problem size would increase and computation time also increase, but the solution could exploit inter-frame correlations, more sparse, and hence yield more accurate reconstructions

5. Problem Formulation Compressive sensing
Compressive coded aperture mask design
Video setting

6. Compressive Sensing
High dimensional vectors can be recovered with high accuracy from a much smaller dimensional observation(y) when f has a “sparse” representation in some basis W.

7. Compressive Coded Aperture Mask
Coded aperture imaging first desires to increase the light hitting a detector without sacrificing resolution. The basic idea is to create a mask pattern which introduces a more complicated point spread function, and exploit this pattern to reconstruct high-quality image estimates.
A recent study addressed the accurate reconstruction of a high resolution static image which has a sparse representation in some basis from a single low resolution observation using compressive coded aperture imaging.
“Compressive coded aperture superresolution image reconstruction”, ICASSP , 2008

8. Compressive Coded Aperture Mask
Observation y is assumed： y = D(f * h) + n D is a downsampling operator and h is a point-spread function (PSF) can also be recognized as coded aperture patterns.
Let F be the matrix corresponding to fast Fourier transform, and denote the Fourier transform of h by H (F(h) = H.), Let be a diagonal matrix whose diagonal components are the entries in H.
Our goal is to design a mask pattern h such that the resulting image reconstruction is better than using no mask. This involves defining a pattern h such that the corresponding observation matrix R satisfies an RIP.

9. Compressive Coded Aperture Mask
A method(by author) for randomly generating a mask h was developed so that the corresponding matrix product is block-circulant:
Block-circulant matrices are known to be a compressed sensing matrix, and based upon recent theoretical work on Toeplitz-structured matrices for compressive sensing, the proposed masks are fast and memory-efficient to compute.

10. Video Setting
The problem of accurately reconstructing a high resolution video from low resolution observations can be modeled mathematically by solving
For judiciously chosen coded aperture masks p such that the corresponding observation matrix R satisfies the RIP, then solving a minimization problem will yield highly accurate reconstructions according to compressive sensing theory.

11. Sparse Representation Algorithm
Our goal is to solve efficiently for a sequence of closely related video frame images
Method A. For a scene that changes only slightly from frame to frame like adjacent frames, the reconstruction from a previous frame is often a good approximation to the following frame. So we use the solution at the frame as the initial value of for the optimization problem for the frame. Thus, few iterations will be needed for each optimization problem.

12. Sparse Representation Algorithm
Method B and C method B and C are both improve method A by solving multiple frames simultaneously
Solving two frames(method B2 and C2) The objective function is separable, two solutions and are not related.

13. Sparse Representation Algorithm
Solve the couple of optimization instead：
We compute and such that , then since , is very sparse compare to

14. Sparse Representation Algorithm
Solving k>2 frames simultaneously in methods B and C Methods B and C are differ by and

15. Sparse Representation Algorithm
Method B4 (4 frames)

16. Sparse Representation Algorithm
Method C4 (4 frames) Sparsity may greater than B4

17. Simulations Optimization problems solving ：GPSR algorithm
X256 gray-scale frames video depicting the movement of a vehicle on a street at the Duke University campus, but campus background remaining stationary.
Low-resolution video image：each frame is downsampled by four in each dimension (resolution=64X64)
Solving different number of frames : Methods A (1 frame), B2/C2 , B4 and C4 ,C8 and C12
Noise variance σ = 1.0 and 0.1

18. Simulations

19. Simulations
Time limited for each frame：5 seconds and 20 seconds(only for σ=1.0)
Observation matrix R and the basis matrix W multiplication take most of the computation time
Method
Iterations in 5 second/frame A
110
B2/C2 48
B4/C4 22
B8/C8 9
C12 4

20. Simulations

21. Conclusion
This paper present compressive sensing methods for overcoming the pixel-limited resolution of digital video imaging systems.
This method apply coded mask designs for each video frame and use sparse representation optimization techniques for signal recovery.
If the desired accuracy is fixed, processing time required to achieve that accuracy is smaller when the block size (number of frames ) is larger, demonstrates the importance of exploiting inter-frame correlations, even when it increasing the size of the problem.