1. IEEE TCSVT 2011 Wonjun Kim Chanho Jung Changick Kim
Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes
IEEE TCSVT 2011
Wonjun Kim
Chanho Jung
Changick Kim

2. Outline Introduction Proposed Method Experiment Result Application
Conclusion

3. Introduction
Problem occurs when background is highly textured

4. Feature Representation
Proposed Method
Feature Representation
Edge orientation histogram (EOH)
Color orientation histogram (COH)
Temporal Feature
Self-ordinal Measure
Saliency Map
Scale-invariant Saliency Map

5. Edge Orientation Histogram (EOH)
Compute the edge orientation of every pixel in the local region center at the 𝑖 𝑡ℎ pixel
Quantized into K angle in the range of [ 0 ° , 180 ° ]
Compute the histogram of edge orientation
local region 𝑖
m(x,y,n):edge magnitude
𝜽(x,y,n):quantized orientation

6. Color Orientation Histogram (COH)
Quantize the angle in HSV color space in the range of [ 0 ° , 360 ° ] into H angles
Compute the histogram of color orientation
Add an additional bin with q=0 for both EOH and COH to handle pixels with zero edge magnitudes and color attributes
s(x,y,n):saturation value
𝒗(x,y,n):quantized hue value

7. Temporal Feature Compute the intensity differences between frames
Feature at the 𝑖 𝑡ℎ pixel of 𝑛 𝑡ℎ frame
P :total number of pixels in local region
j :index of those pixels in P
𝝉 :user-defined latency

8. Self-ordinal Measure
Define a 1×(K+1) rank matrix by ordering the elements of EOH(COH) ex:

9. Self-ordinal Measure

10. Saliency Map of Edge and Color
Compute the distance from the rank matrix of center region to surrounding regions
Saliency Map of Edge
Saliency Map of Color
N :total number of local regions in a center-surround window
𝒁 𝑬 , 𝒁 𝑪 :maximum distance between two rank matrices

11. Spatial Saliency Map
Combine the edge and color saliency

12. Combining with Temporal Saliency
Compute the SAD of temporal gradients between center and the surrounding regions
Combine the spatial and temporal saliency

13. Scale-invariant Saliency Map
Combine 3 different scales of saliency Map (32×32, 64×64, 128×128)
32×32
64×64
128×128

14. Algorithm

15. Experiment Result
Static Images
Video Sequences

16. Experiment Result Static Image Video Sequence Local region = 5×5
center-surround window = 7×7
K = 8, H= 6
𝑍 𝐸 = 40, 𝑍 𝐶 = 24
Video Sequence
𝑍 𝑇 = 49
Speed: 23ms per frame (43 fps)

17. Static Images

18. Static Images

19. Video Sequences

20. Video Sequences

21. Moving Object Extraction
Application
Image Retargeting
Moving Object Extraction

22. Image Retargeting

23. Image Retargeting

24. Moving Object Detection
G:the set of salient pixels in the ground truth image
P:salient pixels in the binarized object map
Card(A):the size of the set A
When the scene is cluttered or the background is complex some old methods might lead to severe distortions

25. Moving Object Detection

27. Conclusion
Ordinal signature can tolerate more local feature distribution than sample values.
The proposed scheme performs in real-time and can be extended in both static and dynamic scenes.