1. Progressive Transmission and Rendering of Foveated Volume Data
Chen Chen, Zhiyong Huang, Hang Yu, Ee-Chien Chang
School of Computing
National University of Singapore

2. Introduction - motivation
Problem
Large volume data provided by modern scanners like CT and MRI
Slow network speed comparing to the large size of volume data
Solution
Wavelet foveation
Progressive transmission and rendering
coarse-to-fine
region-based
2018/11/19
grapp 2006

3. Introduction - wavelet foveated volume
Foveated volume is a non-uniform sampled volume whose resolution is highest at the fovea (user specified ROI) and falls off as the distance to the fovea increases – priority selection
Wavelet foveated volume is constructed by select ROI at each level of details in the volume’s wavelet form - compression
2018/11/19
grapp 2006

4. Proposed method - overview
Transform to Wavelet Form
Divide into 83 block, run-length encode (RLE) each block
Client request = ROI {(x,y,z),s}
Generate Fw
Reorder data blocks of Fw
Progressive transmission
Progressive Rendering
2018/11/19
grapp 2006

5. Proposed method - overview
Client-server volume rendering system
Server Side:
Input: Cw & ROI{(x,y,z),s}
Output: Fw
Client Side:
Input: Fw
Output: Iw1 -> Iw2 -> … -> Iwfinal
Two transmission and rendering schemes
2018/11/19
grapp 2006

6. Proposed method - overview
Server Site
Client Site
Wavelet Transform Cw
request
Generate Fw from
Cw and ROI
User Input
ROI={(x,y,z),r}
Reorder data blocks in Fw
according to the schemes
Progressive Render
RB or CTF
Progressive
transmission
Progressive
rendering
Display
2018/11/19
grapp 2006

7. Wavelet transform Divide the volume into blocks of size 2k3
Apply generic Haar wavelet transform on each block to obtain 7 detail blocks and 1 average block
Group 8 adjacent blocks to again obtain a block of size 2k3
Repeat this procedure until the average volume reaches size m3
2018/11/19
grapp 2006

8. Wavelet transform - run-length encoding (RLE)
A more compact representation
RLE is performed block by block
Compressed block will be a list of value(v) and length(l) pairs
v= Integer value of a coefficient l = Number of successive v
Example:
L=1,1,1,1,0,0,5,0,0,0,0,0,0,0,0
R=(1,4),(0,2),(5,1),(0,8)
2018/11/19
grapp 2006

9. Derive foveated volume
Cw, ROI{(x,y,z),s} => Fw
Layer 0 image = average image of Cw
Layer i image = IWT(layer i-1 image, 7 layer i-1 detail images)
ROI in ith layer = ROI in Layer i image
2018/11/19
grapp 2006

10. Derive foveated volume
Center of ROI in ith layer is
Size of ROI in ith layer is s3
Boundary of ROI in ith layer can be defined as
2018/11/19
grapp 2006

11. Derive foveated volume
4th ROI = {(9,9),6} = {[6,11],[6,11]}
Layer 3 details (i,j) =([3,5],[3,5])
2018/11/19
grapp 2006

12. Progressive transmission - motivation
Data size = 10243, Fovea size = 483, Average image size = 163
Maxlayer=Log2(1024/16)=6
Total amount of data = 243*(5*7+1)=497,664 coefficients
Reorder data blocks
Progressive transfer
2018/11/19
grapp 2006

13. Progressive transmission - two schemes
Coarse-to-fine
Description: Transmit a rough average image first and refine the fovea from outer ROIi to inner ROIi+1
Implementation: Send the wavelet foveated data from inner layer to outer layer progressively
Region-based
Description: Transmit full resolution ROImaxlayer first and expand the image from ROImaxlayer-1 to ROI0 with decreasing resolution
Implementation: Data blocks and parent data blocks of ROImaxlayer-i+1 will be sent in ith iteration
2018/11/19
grapp 2006

14. Progressive transmission - illustration in 2D
2D image size= 128*128, m=16, k=8, ROI={(86,86),45}, maxlayer=3
ROI0={(10,10),45}=[0,15] blk_index=[0,1]
ROI1={(21,21),45}=[0,31] blk_index=[0,1]
ROI2={(43,43),45}=[20,63] blk_index=[1,3]
ROI3={(86,86),45}=[64,107] blk_index=[4,6]
2018/11/19
grapp 2006

15. Progressive transmission - illustration in 2D
2018/11/19
grapp 2006

16. Progressive transmission - coarse-to-fine
2018/11/19
grapp 2006

17. Progressive transmission - coarse-to-fine
2018/11/19
grapp 2006

18. Progressive transmission - region-based
2018/11/19
grapp 2006

19. Progressive rendering - rendering equation
For each voxel:
intensity value
opacity value
The final intensity result reaches the viewer of each pixel will be Eulerian Sum over accumulated opacity
Rendering equation to render a subvolume with same intensity and opacity value
2018/11/19
grapp 2006

20. Progressive rendering - rendering equation
For 2 subvolumes Sa and Sb with size na and nb, voxels in Sa and Sb have , , and , respectively. Sa is in front of Sb, that is, Sa is “over” Sb. The intensity of Sa and Sb and Sfinal can be defined as
2018/11/19
grapp 2006

21. Region-based - partial volume reconstruction
Starting coefficient:
Ending coefficient:
Wavelet coefficients at ith layer detail image
begin:
end:
Reconstruction starts from layer 0, that is, the average image and move out to higher level of details until the desired resolution is reached
2018/11/19
grapp 2006

22. Region-based
Center fovea is reconstructed using partial volume reconstruction and rendered at iteration 1
6 subvolumes are rendered: Top, Bottom, Left, Right, Front, Back with resolution scale 2(1-i), i is the number of iteration
6 sub volumes are rendered separately and combine the rendered result to the previous rendering image
2018/11/19
grapp 2006

23. Region-based - image composition
Top, bottom, left and right are added to the previous rendering result image at their right position
Front and back image needs image composition with the previous rendering result image: (front) over (inner) over (back)
“over” operator can be defined as: given image pixel A(da, αa) and B(db, αb)
C = A over B
dc = da + αa*db
αc = αa* αb
2018/11/19
grapp 2006

24. Region-based - image composition
2018/11/19
grapp 2006

25. Coarse-to-fine Start rendering from ROI0 to ROImaxlayer
Each ROIi except ROImaxlayer can be divided into 7 subvolumes: top, bottom, left, right, front, back and center
Center subvolume is the average image of the inner ROI and its rendering result, accumulated density value and opacity value will be kept for next iteration
2018/11/19
grapp 2006

26. Coarse-to-fine
ROIi is constructed by center part of ROIi-1 and level i detail coefficients
Number of voxels each coefficient in ROIi represent is 2maxlayer-i
Rendering equation for each voxel in ROIi is
Rendering result of front and back subvolume RFi and RBi will be take down for next iteration
Rendering result of iteration i+1 will be combined with RFi and RBi in iteration i+1
2018/11/19
grapp 2006

27. Results - direct rendering
2018/11/19
grapp 2006

28. Results - region-based
2018/11/19
grapp 2006

29. Results - region-based
2018/11/19
grapp 2006

30. Results - coarse-to-fine
2018/11/19
grapp 2006

31. Results - coarse-to-fine
2018/11/19
grapp 2006

32. Results time
2018/11/19
grapp 2006

33. Results - coarse-to-fine
2018/11/19
grapp 2006

34. Results - coarse-to-fine
2018/11/19
grapp 2006

35. Results - region-based
2018/11/19
grapp 2006

36. Results - region-based
2018/11/19
grapp 2006

37. Results - coarse-to-fine
2018/11/19
grapp 2006

38. Results - coarse-to-fine
2018/11/19
grapp 2006

39. Results - region-based
2018/11/19
grapp 2006

40. Results - region-based
2018/11/19
grapp 2006

41. Conclusion and future work
Two progressive transmission and rendering schemes, region-based and coarse-to-fine
Blockwise RLE used in wavelet compression
Wavelet foveation
Image composition
Large memory is still needed
cut the data set exceeding a certain amount into smaller data sets and build location map to indicate actual position of each data segments
Cache can be used to store frequently used blocks, for example, those high level details
2018/11/19
grapp 2006