1. Deep Video Quality Assessor: From Spatio-temporal Visual Sensitivity to A convolutional Neural Aggregation Network
Woojae Kim1, Jongyoo Kim2, Sewoong Ahn1,Jinwoo Kim1, and Sanghoon Lee1
1Department of Electrical and Electronic Engineering, Yonsei University
2Microsoft Research, Beijing, China
2019/10/26
Yuwen Li

2. Motivations Temporal motion effect: i)temporal masking effect;
ii)a severe error
in the motion
map makes
spatial errors
more visible to
humans
2019/10/26
Yuwen Li

3. Motivations
Temporal memory for quality judgment
2019/10/26
Yuwen Li

4. Contributions
A Deep Video Quality Assessor (DeepVQA) to predict the spatio- temporal sensitivity map
A Convolutional Neural Aggregation Network (CHAN) borrowing an idea from an 'attention mechanism'
2019/10/26
Yuwen Li

5. Related Works Spatio-temporal Visual Sensitivity:
i)A spatio temporal contrast sensitivity function (CSF)
ii)A natural video statistics (NVS) theory
iii)Existing attempts using deep learning failed to consider motion properties.
Temporal Pooling:
i)Average pooling
ii)Adaptively pool the temporal scores from the HVS perspective
iii)'Neural Aggregation Network for Video Face Recognition' (CVPR2017)
2019/10/26
Yuwen Li

6. Framework
2019/10/26
Yuwen Li

7. Framework-Step 1 Input: Distorted frame:
normalized after subtracting the lowpass
filtered frames
Spatial Error map:
2019/10/26
Yuwen Li

8. Framework-Step 1 Input: Frame Difference map: Temporal Error map:
2019/10/26
Yuwen Li

9. Framework-Step 1 Intermediate output: Spatio-temporal Sensitivity map:
2019/10/26
Yuwen Li

10. Framework-Step 1 Intermediate output: Perceptual Error map: 2019/10/26
Yuwen Li

11. Framework-Step 1
2019/10/26
Yuwen Li

12. Framework-Step 2
2019/10/26
Yuwen Li

13. Experiments
2019/10/26
Yuwen Li

14. Experiments
2019/10/26
Yuwen Li

15. Experiments
2019/10/26
Yuwen Li

16. Experiments
2019/10/26
Yuwen Li

17. Experiments
2019/10/26
Yuwen Li

18. Experiments
2019/10/26
Yuwen Li

19. Experiments
2019/10/26
Yuwen Li

20. Conclusion +How to tell a good story
-Act like an integration of their previous work
-Generalization ability
-Hard to transform to NR-VQA
2019/10/26
Yuwen Li

21. References
Yang, J., Ren, P., Zhang, D., Chen, D., Wen, F., Li, H., Hua, G., Yang, J., Li, H.,Dai, Y., et al.: Neural aggregation network for video face recognition. In: Proc.IEEE Conf. Comput. Vis. Pattern Recognit.(CVPR). 2492–2495
Kim, J., Lee, S.: Deep learning of human visual sensitivity in image quality assessment framework. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit.(CVPR).(2017)
2019/10/26
Yuwen Li