Deep Video Quality Assessor: From Spatio-temporal Visual Sensitivity to A convolutional Neural Aggregation Network Woojae Kim1, Jongyoo Kim2, Sewoong Ahn1,Jinwoo.

Slides:

Advertisements

Similar presentations

Adaptive Offset Subspace Self- Organizing Map with an Application to Handwritten Digit Recognition Huicheng Zheng, Pádraig Cunningham and Alexey Tsymbal.

Advertisements

Introduction to Image Quality Assessment

1 Blind Image Quality Assessment Based on Machine Learning 陈欣

Spatial Pyramid Pooling in Deep Convolutional

Perceived video quality measurement Muhammad Saqib Ilyas CS 584 Spring 2005.

Hurieh Khalajzadeh Mohammad Mansouri Mohammad Teshnehlab

Video Tracking Using Learned Hierarchical Features

University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.

MULTIMEDIA INPUT / OUTPUT TECHNOLOGIES

Department of computer science and engineering Evaluation of Two Principal Image Quality Assessment Models Martin Čadík, Pavel Slavík Czech Technical University.

Hierarchical Matching with Side Information for Image Classification

Skeleton Based Action Recognition with Convolutional Neural Network

Journal of Visual Communication and Image Representation

Marcus Barkowsky, Savvas Argyropoulos1 Towards a Hybrid Model Provide a structure with building blocks Provide a programming and evaluation environment.

Regionlets for Generic Object Detection IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 37, NO. 10, OCTOBER 2015 Xiaoyu Wang, Ming.

Local Stereo Matching Using Motion Cue and Modified Census in Video Disparity Estimation Zucheul Lee, Ramsin Khoshabeh, Jason Juang and Truong Q. Nguyen.

National Taiwan Normal A System to Detect Complex Motion of Nearby Vehicles on Freeways C. Y. Fang Department of Information.

Face Recognition based on 2D-PCA and CNN

Video object segmentation and its salient motion detection using adaptive background generation Kim, T.K.; Im, J.H.; Paik, J.K.; Electronics Letters

Deep Learning for Dual-Energy X-Ray

Faster R-CNN – Concepts

IEEE BIBM 2016 Xu Min, Wanwen Zeng, Ning Chen, Ting Chen*, Rui Jiang*

Data Mining, Neural Network and Genetic Programming

Convolutional Neural Fabrics by Shreyas Saxena, Jakob Verbeek

DNN-Based Urban Flow Prediction

Krishna Kumar Singh, Yong Jae Lee University of California, Davis

Saliency-guided Video Classification via Adaptively weighted learning

Regularizing Face Verification Nets To Discrete-Valued Pain Regression

Combining CNN with RNN for scene labeling (segmentation)

Introductory Seminar on Research: Fall 2017

Mean Euclidean Distance Error (mm)

Multiple Wavelet Coefficients Fusion in Deep Residual Networks for Fault Diagnosis

Human-level control through deep reinforcement learning

CNNs and compressive sensing Theoretical analysis

Introduction to Neural Networks

Tuning JPEG2000 Image Compression for Graphics Regions

Wei Liu, Chaofeng Chen and Kwan-Yee K. Wong

Convolutional Neural Networks for Visual Tracking

Figure 4. Testing minimal configurations with existing models for spatiotemporal recognition. (A-B) A binary classifier is trained to separate a positive.

Two-Stream Convolutional Networks for Action Recognition in Videos

Towards Understanding the Invertibility of Convolutional Neural Networks Anna C. Gilbert1, Yi Zhang1, Kibok Lee1, Yuting Zhang1, Honglak Lee1,2 1University.

Object Detection + Deep Learning

Introduction of MATRIX CAPSULES WITH EM ROUTING

Hairong Qi, Gonzalez Family Professor

8-3 RRAM Based Convolutional Neural Networks for High Accuracy Pattern Recognition and Online Learning Tasks Z. Dong, Z. Zhou, Z.F. Li, C. Liu, Y.N. Jiang,

Single Image Rolling Shutter Distortion Correction

MEgo2Vec: Embedding Matched Ego Networks for User Alignment Across Social Networks Jing Zhang+, Bo Chen+, Xianming Wang+, Fengmei Jin+, Hong Chen+, Cuiping.

Anomaly Detection in Crowded Scenes

Pattern recognition in gait activities using a floor sensor system

An Introduction to Computer Vision& Pattern Recognition Group

Outline Background Motivation Proposed Model Experimental Results

SPM2: Modelling and Inference

Visualizing and Understanding Convolutional Networks

Machine Learning based Data Analysis

Neural Network Pipeline CONTACT & ACKNOWLEDGEMENTS

Spatially Supervised Recurrent Neural Networks for Visual Object Tracking Authors: Guanghan Ning, Zhi Zhang, Chen Huang, Xiaobo Ren, Haohong Wang, Canhui.

Human-object interaction

Visual Manipulation Relationship Network for Autonomous Robotics

Natalie Lang Tomer Malach

VERY DEEP CONVOLUTIONAL NETWORKS FOR LARGE-SCALE IMAGE RECOGNITION

Visual Grounding 专题报告 Lejian Ren 4.23.

Unrolling the shutter: CNN to correct motion distortions

End-to-End Facial Alignment and Recognition

Deep learning: Recurrent Neural Networks CV192

Week 3 Volodymyr Bobyr.

Week 7 Presentation Ngoc Ta Aidean Sharghi

SDSEN: Self-Refining Deep Symmetry Enhanced Network

What and How Well You Performed

Lark Kwon Choi, Alan Conrad Bovik

Presentation transcript:

Deep Video Quality Assessor: From Spatio-temporal Visual Sensitivity to A convolutional Neural Aggregation Network Woojae Kim1, Jongyoo Kim2, Sewoong Ahn1,Jinwoo Kim1, and Sanghoon Lee1 1Department of Electrical and Electronic Engineering, Yonsei University 2Microsoft Research, Beijing, China 2019/10/26 Yuwen Li

Motivations Temporal motion effect: i)temporal masking effect; ii)a severe error in the motion map makes spatial errors more visible to humans 2019/10/26 Yuwen Li

Motivations Temporal memory for quality judgment 2019/10/26 Yuwen Li

Contributions A Deep Video Quality Assessor (DeepVQA) to predict the spatiotemporal sensitivity map A Convolutional Neural Aggregation Network (CHAN) borrowing an idea from an 'attention mechanism' 2019/10/26 Yuwen Li

Related Works Spatio-temporal Visual Sensitivity: i)A spatio temporal contrast sensitivity function (CSF) ii)A natural video statistics (NVS) theory iii)Existing attempts using deep learning failed to consider motion properties. Temporal Pooling: i)Average pooling ii)Adaptively pool the temporal scores from the HVS perspective iii)'Neural Aggregation Network for Video Face Recognition' (CVPR2017) 2019/10/26 Yuwen Li

Framework 2019/10/26 Yuwen Li

Framework-Step 1 Input: Distorted frame: normalized after subtracting the lowpass filtered frames Spatial Error map: 2019/10/26 Yuwen Li

Framework-Step 1 Input: Frame Difference map: Temporal Error map: 2019/10/26 Yuwen Li

Framework-Step 1 Intermediate output: Spatio-temporal Sensitivity map: 2019/10/26 Yuwen Li

Framework-Step 1 Intermediate output: Perceptual Error map: 2019/10/26 Yuwen Li

Framework-Step 1 2019/10/26 Yuwen Li

Framework-Step 2 2019/10/26 Yuwen Li

Experiments 2019/10/26 Yuwen Li

Experiments 2019/10/26 Yuwen Li

Experiments 2019/10/26 Yuwen Li

Experiments 2019/10/26 Yuwen Li

Experiments 2019/10/26 Yuwen Li

Experiments 2019/10/26 Yuwen Li

Experiments 2019/10/26 Yuwen Li

Conclusion +How to tell a good story -Act like an integration of their previous work -Generalization ability -Hard to transform to NR-VQA 2019/10/26 Yuwen Li

References Yang, J., Ren, P., Zhang, D., Chen, D., Wen, F., Li, H., Hua, G., Yang, J., Li, H.,Dai, Y., et al.: Neural aggregation network for video face recognition. In: Proc.IEEE Conf. Comput. Vis. Pattern Recognit.(CVPR). 2492–2495 Kim, J., Lee, S.: Deep learning of human visual sensitivity in image quality assessment framework. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit.(CVPR).(2017) 2019/10/26 Yuwen Li