1. Ali Ercan & Ulrich Barnhoefer
Motion Compensated SNR and DR Enhancement With Motion Blur Prevention Using Multicapture
Ali Ercan & Ulrich Barnhoefer

2. Introduction & Motivation
Single exposure trade-off
High noise if short exposure time
Motion blur if long exposure time
When presenting the example mention
Simulated scene at Matlab
Camera moving
Short exposure: Shot and read noise
Long exposure: Motion blur
EE392J Project
Ali Ercan & Ulrich Barnhoefer

3. Introduction & Motivation
DR is another problem
Short exposure: Dark areas in the scene cannot be seen
Long exposure: Bright areas saturate
If both high DR scene and motion, with single capture
Motion blur free, but noisy and non-visible dark areas image
Less noisy, but motion blurred and saturated image
EE392J Project
Ali Ercan & Ulrich Barnhoefer

4. Introduction & Motivation
Our approach to solve these problems:
Use of multicapture combined with motion estimation
High speed is definitely needed
Normal video mode can be used – poorer results due to noise adding
CMOS imagers suitable
For a better understanding, let us introduce a simple model of CMOS imagers and describe multicapture
EE392J Project
Ali Ercan & Ulrich Barnhoefer

5. Ali Ercan & Ulrich Barnhoefer
Sensor Model
Charge Integration
Light on photodiode generates charges
Saturation when well capacity is reached
Noise sources
(Reset noise)
Shot noise UT
Read noise VT,Vo
(Dark current)
(Fixed pattern noise)
- Charge integrates on intrinsic capacitor as light shines on photodiode
- Maximum charge proportional to reset voltage - saturation
CDS
Equation: Signal + noise
EE392J Project
Ali Ercan & Ulrich Barnhoefer

6. Ali Ercan & Ulrich Barnhoefer
Multicapture
Nondestructive multiple readout – Single integration
Less noise per capture compared to conventional video sensor – readout noise
cumulative read out noise in a conventional video sensor
EE392J Project
Ali Ercan & Ulrich Barnhoefer

7. Implemented Algorithm
SCENE
CAMERA
SIMULATOR
MOTION
ESTIMATOR
PHOTO-CURRENT
FINAL
IMAGE
EE392J Project
Ali Ercan & Ulrich Barnhoefer

8. Ali Ercan & Ulrich Barnhoefer
Camera Simulator
Multicapture, noise, ADC implemented – pixel values out
Synthetic scene
Black background – to see the effect of read noise only, will be clear later why
Camera moves
EE392J Project
Ali Ercan & Ulrich Barnhoefer

9. Ali Ercan & Ulrich Barnhoefer
Motion Estimator
Block based motion estimation on difference frames
Search range ±1 and block size 3x3
Fast imager (e.g. 10,000 fps available)
Search range and block size can be increased in expense of computational load
Noise suppression
Known noise levels – characterized CMOS sensor
Error = SSD + xDistance
 is proportional to noise
Thanks to Sebe!
EE392J Project
Ali Ercan & Ulrich Barnhoefer

10. Ali Ercan & Ulrich Barnhoefer
Motion Estimator
Estimated and perfect motion vectors
EE392J Project
Ali Ercan & Ulrich Barnhoefer

11. Photocurrent EstimatorB C
EE392J Project
Ali Ercan & Ulrich Barnhoefer

12. Photocurrent Estimator
Useful when you have read noise
When only shot noise is present, take the last Q_tilda
EE392J Project
Ali Ercan & Ulrich Barnhoefer

13. Ali Ercan & Ulrich Barnhoefer
Results
EE392J Project
Ali Ercan & Ulrich Barnhoefer

14. Ali Ercan & Ulrich Barnhoefer
Results
EE392J Project
Ali Ercan & Ulrich Barnhoefer

15. Ali Ercan & Ulrich Barnhoefer
Results
Emphasize we get perfect result even with not perfect motion vectors
Ambiguity means lighting did not change
Symmetry of the scene
EE392J Project
Ali Ercan & Ulrich Barnhoefer

16. Results IMAGE ERRORS (STD OF ERROR IMAGE) CHECKER LENA CAMERAMAN
10 ms image
100.9
69.43
71.31
160 ms image
70.79
33.84
37.41
Const. with est. motion vectors
2.587
21.28
12.05
Const. with perfect motion vectors
2.576
17.22
3.092
EE392J Project
Ali Ercan & Ulrich Barnhoefer

17. Ali Ercan & Ulrich Barnhoefer
Conclusion
Promising results achieved with this preliminary analysis
Motion blur reduced
Noise reduced
DR increased in dark end and in bright end in special cases
EE392J Project
Ali Ercan & Ulrich Barnhoefer

18. Ali Ercan & Ulrich Barnhoefer
Conclusion
Lots of more things to do
Use more sophisticated motion estimation algorithms
Separate motion detection from motion estimation and do estimation when detection occurs
Include extension of DR with sensor saturation
Handle the occlusions
EE392J Project
Ali Ercan & Ulrich Barnhoefer

19. Ali Ercan & Ulrich Barnhoefer
Questions
EE392J Project
Ali Ercan & Ulrich Barnhoefer