1. Improved Census Transforms for Resource-Optimized Stereo Vision
Wade S. Fife, Member, IEEE,
James K. Archibald, Senior Member, IEEE
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 23, NO. 1, JANUARY 2013

2. Outline Introduction Related Work Proposed Algorithm
Sparse Census Transform
Generalized Census Transform
Hardware Implementation
Experimental Results
Conclusion

3. Introduction

4. Introduction The challenges: It is critical to…
The enormous amount of computation required to identify the corresponding points in the images.
It is critical to…
maximize the accuracy and throughput of the stereo system
while minimizing the resource requirements

5. Objective Propose the sparse census transforms :
Reduce the resource requirements of census-based systems
Maintain correlation accuracy
Propose the generalized census transforms :
A new class of census-like transforms
Increase the robustness and flexibility

6. Related Work

7. Related Work
Census Transform :
Color
Gradient

8. Related Work After aggregation step: Census on colors
Census on gradients

9. Related Work Sparse census[6] : Half of the bits
The computation costs for the hamming distances are quite large. X
[6] C. Zinner, M. Humenberger, K. Ambrosch, and W. Kubinger, “An optimized software-based implementation of a census-based stereo matching algorithm,” in Proc. 4th ISVC, 2008, pp. 216–227.

10. Related Work Mini-census[8] : X
[8] N.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen, and T.-S. Chang,“Algorithm and architecture of disparity estimation with mini-census adaptive support weight,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 792–805, Jun

11. Related Work Mini-census[8] : Mini-census adaptive support weight
[8] N.-C. Chang, T.-H. Tsai, B.-H. Hsu, Y.-C. Chen, and T.-S. Chang,“Algorithm and architecture of disparity estimation with mini-census adaptive support weight,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 792–805, Jun

12. Related Work
Mini-census[8] :

13. Proposed Algorithm

14. Sparse Census Transform
Definition :
N: the set of points within a T × T window around p
𝑁 : a new set of N P’ P

15. Transform Point Selection
Goal : minimize the size of the census transform vector
Challenge: Must quantify how much each point in the transform window contributes to overall correlation accuracy
Test correlation accuracy:
Define a sparse census transform consisting of a single point (| 𝑵 | = 1)
Determine how consistently this point leads to correct correlation
13 × 13 correlation window (aggregation)

16. Transform Point SelectionGo
Tsukuba
Venus
Average
Bright:
Higher correlation accuracy
25 × 25 neighborhood
Teddy
Cones

18. Transform Point Selection
Further from the center : value decreasing
Very near the center : less effective
It is best to choose points that are neither too far from nor too close to the center pixel.
Optimal distance : 2 pixels
If the image is noisy should be slightly further
from the center

19. Transform Point Selection
With Gaussian noise
( 𝝈 = 5.12)
Tsukuba
Tsukuba
Venus
Venus
Average
Bright:
Higher correlation accuracy
37 × 37 neighborhood
Teddy
Teddy
Cones
Cones

20. Proposed Sparse Census Transform
Very good correlation accuracy can be achieved using very sparse transforms.
16-point
12-point
8-point
4-point
2-point
1-point

21. Experimental Results

22. Generalized Census Transform
Goal : greater freedom in choosing the census transform design
Definition : redrawing the transform as a graph
3 × 3 correlation
(aggregation)
3 × 3 census

23. Generalized Census Transform
As..
(1)transform neighborhoods become more and more sparse
(2)fewer pixels are used in the correlation process
selection of points to include in the transform becomes more critical
Horizontal + Vertical
2-edge
2-point

24. Generalized Census Transform
symmetric

25. Proposed Generalized Census Transform
Benefits :
Often require a smaller census transform window (memory)
Increased robustness under varying conditions (noise)
16-edge
12-edge
8-edge
4-edge
2-edge
1-edge

26. Experimental Results

27. Experimental Results

28. Hardware Implementation
Pipelining : to increase throughput in an FPGA implementation
(Field Programmable Gate Array)
One input pixel per clock cycle &
Output one disparity result per clock cycle
Range : 0~3

29. Hardware Implementation
Correlation window sum (Aggregation) :

30. Experimental Results

31. Left Image
Ground Truth
Full 7x7 census
12-edge
4-edge

32. Left Image
Ground Truth
Full 7x7 census
12-edge
4-edge

33. Left Image
Full 7x7 census
12-edge
4-edge

34. Experimental Results
𝟖𝟖%↓
𝟔𝟏%↓
LUTs (look-up tables) : the amount of logic required to implement the method
FFs : the number of 1-bit registers (the amount of pipelining used)
RAMs : the number of 18-kbit block memories
Freq. : the maximum operating frequency reported by synthesis

35. Conclusion

36. Conclusion Proposed and analyzed in this paper:
A range of sparse census transforms
reduce hardware resource requirements
attempting to maximize correlation accuracy.
often better than or nearly as good as the full census
Generalized census transforms
increased robustness in the presence of image noise