1. Image Processing in the block DCT Space
Jayanta Mukhopadhyay
Department of Computer Science & Engineering
Indian Institute of Technology, Kharagpur, , India

2. Processing in the spatial domain

3. Processing in the compressed domain

4. Motivations Computation with reduced storage.
Avoid overhead of inverse and forward transform.
Exploit spectral factorization for improving the quality of result and speed of computation.

5. Typical Applications
Resizing.
Transcoding.
Enhancement.
Retrieval.

6. Typical Applications
Compressed image streams with varying resolutions may be transmitted over channels of varying bandwidth
Browsing of images in different resolutions over internet, eg. Initially at low resolution and later, if interested, browsed with higher resolution.

7. DCT Definitions

8. Different types of DCTs
Let x (n), n=0,1,2,...N+1 be a sequence of length N+1. Then N-point type-I DCT is defined as:
Note:- Type-I N-point DCT is defined over N+1 data points.

9. Different types of DCTs
Similarly, x (n), n=0,1,2,...N, be a sequence of length N. Its N-point type-II DCT is defined as:
Note:- Type-II N-point DCT is defined over N data points.

10. where is given by the following equation:

11. Sub-band Relationship

12. Subband DCT DCT of a 2D images x(m,n) of size N x N
Jung, Mitra and Mukherjee , IEEE CSVT (1996)‏

13. Sub-band approximation
Low-low subband
Sub-band approximation

14. Truncated approximation
Generalized Truncated approximation
C′ (k,l) = √(L.M).C(k,l), k,l=0,1,...., (N-1)
= otherwise
Mukherjee and Mitra (2005), IEE VISP

15. Spatial Relationship

16. Need for Analysis of the Relation between DCT blocks and sub-blocks
Direct conversion of DCT blocks of one size into another possible in the compressed domain itself
Saving of computation complexity and memory required
Decompression and re-compression steps reduced
DCT matrices being sparse number of multiplications and additions reduced
DCT blocks of different sizes required for different standards
Jiang and Feng (2002), IEEE SP

17. Spatial Relationship of DCT coefficients

18. Standard N-point DCT of B={x(i)} and of its pth section Tp

19. Basis function b(k,t) constructed from b1(k,t)‏

20. Equations to find the transform matrices for composition and decomposition

21. EXAMPLES OF TRANSFORM MATRICES –
– –0.0690
– – –
A(2,2)= –

22. Block Composition and Decomposition in 2D
Composition of L x M adjacent N x N DCT blocks to a single LN x MN DCT block:
Decomposition of LN x MN size DCT block to L x M adjacent N x N DCT blocks

23. Example of the conversion matrix
Note:-
The conversion matrices and their inverses are sparse.
Requires less number of multiplications and additions than those of usual matrix multiplications.

24. DCT domain Filtering

25. Type-I Symmetric Extension
For type-I DCT symmetric extension of N+1 sample take place in the following:

26. Type-II Symmetric Extension
For type-II DCT the symmetric extension of input sequence is done in the following manner:

27. Symmetric Convolution
Let type –II symmetric extension of x(n) be
denoted by and type –I symmetric
extension of h(n) be denoted as
Symmetric convolution of x(n) and h(n) defined as follows:

28. Symmetric convolution
h(n)
x(n)
y(n)=h(n)*x(n)
HI(k)
XII(k)
YII(k)=HI(k).XII(k)
Symmetric convolution
operation
CONVOLUTION-MULTIPLICATION PROPERTY
Martucci (1994), IEEE SP

29. Filtering Approaches Based on Symmetrical Convolution
Linear filtering with DCT of filter matrices

30. 1. Filtering with Symmetric Convolution
Note:- JPEG compression scheme uses type-II DCT, transform domain filtering directly gives the type-II DCT.
Martucci and Mersereau, IEEE ASSSP 1993,
Martucci, IEEE SP 1994,
Mukherjee and Mitra LNCS 2006

31. BOUNDARY EFFECT IN BLOCK DCT DOMAIN
Linear Convolution
Block Symmetric
Convolution

32. Computation with larger blocks: The BFCD filtering Algorithm
Original Image
8x8 blocks
Block
Composition
Decomposition
Filtered Image
Original image
Larger DCT blocks
Filtered image
Convolution
Multiplication
Filtering
Operations
( Mukherjee and Mitra, LNCS-4338, ICVGIP 2006)

33. Exact Computation: The Overlapping BFCD (OBFCD) filtering Algorithm
Original Image
8x8 blocks
Block
Composition
Decomposition
Filtered Image
Original image
Larger DCT blocks
Filtered image
Convolution
Multiplication
Filtering
Operations
Accept
Central
Blocks
( Mukherjee and Mitra, LNCS-4338, ICVGIP 2006)

34. Filtered Images
BFCD Algorithm (5x5)
OBFCD Algorithm

35. Computational Cost: BFCD
310.5
71.5 M + 96 A 4
191.84
46.17 M A 3
116
28 M + 32 A 2
Equivalent
Operations
Per Pixel
Computation
L(=M)

36. Computational Cost: OBFCD
614.76
M A 5
552
M A 4
431.64
M A 3
Equivalent
Operations
Per Pixel
Computation
L(=M)

37. Removal of Blocking Artifacts
Compressed with quality factor 10
Artifacts removed
By filtering in DCT
domain

38. Image Sharpening in DCT domain
Lena
Peppers

39. 2. Linear Filtering in the block DCT domain
Here, both filter matrices and input both in Type-II DCT space.
Spatial Domain:
DCT Domain:
Merhav and Bhaskaran, HPL-95-56, 1996, Kresch and Merhav, IEEE IP 1999, Yim, IEEE CSVT 20004

40. Filtering in the block DCT domain
DCT convolution-multiplication property reduces the cost of filtering.
Type-I DCT of the filter and the Type-II DCT data are used.
To avoid boundary effect, Filtering is performed on the larger DCT block.
The filtering technique by Mukherjee and Mitra
Three separate steps: composition, filtering and decomposition of the DCT blocks.
Cost of filtering is high.
In this work, three operations are computed in a single step.
( Mukherjee and Mitra, LNCS-4338, ICVGIP 2006)

41. CMP Filtering: an improvement
B_1 B0 B1
Input
1. Compose
2. Filter
3. Decompose
Filter response
Point wise Multiply
Type-II DCT, B
Type-I DCT of Filter
Type-II DCT, Bf
Bf-1
Bf 0
Bf 1
Steps 1-3: combined step

42. Filtering in the block DCT space1. 2. 3. 4. 5.

43. Filtering in the block DCT space
In 2-D:

44. Gaussian Filtering…

45. Results: Gaussian filter
Close to 300 dB
4x4
Note:- Sparse DCT block

46. Performance: Gaussian filtering
Variation in
Number of blocks
No variation
in PSNR and cost
Cost varies with L and M

47. Threshold variation

48. Performance of proposed filtering for noisy images

49. Results:Threshold variation
Original Images
Threshold =10-3
Threshold =10-4

50. Blocking artifacts removal

51. Image Sharpening
Sharpened
Images

52. Observations
The proposed technique provides equivalent results as obtained by Linear convolution.
Complexity can be reduced by varying the threshold value.
The technique is comparable with existing techniques.
The technique is based on simple linear operations.

53. Thanks

54. where,
Type I 2D-DCT
Type-I 2-D DCT of a block

55.   Type II 2D-DCT     X k ,     Type-II 2-D DCT of a blockN  1 N  1            ( k )‏  (  )‏ x m , n
cos 2 m  1    k
cos 2 n  1   ,
where, N 2 N 2 N m  n   k ,   N  1