Image Inpainting Using Pre-Trained Classification CNN

Slides:

Advertisements

Similar presentations

World of work How is science used in the world of work? Science in the world of work.

Advertisements

1 Image Classification MSc Image Processing Assignment March 2003.

Unsupervised Learning With Neural Nets Deep Learning and Neural Nets Spring 2015.

1 Automated Feature Abstraction of the fMRI Signal using Neural Network Clustering Techniques Stefan Niculescu and Tom Mitchell Siemens Medical Solutions,

Spatial Pyramid Pooling in Deep Convolutional

Image Denoising and Inpainting with Deep Neural Networks Junyuan Xie, Linli Xu, Enhong Chen School of Computer Science and Technology University of Science.

CS532 Neural Networks Dr. Anwar Majid Mirza Lecture No. 3 Week2, January 22 nd, 2008 National University of Computer and Emerging.

Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona.

Hurieh Khalajzadeh Mohammad Mansouri Mohammad Teshnehlab

KYLE PATTERSON Automatic Age Estimation and Interactive Museum Exhibits Advisors: Prof. Cass and Prof. Lawson.

Representations for object class recognition David Lowe Department of Computer Science University of British Columbia Vancouver, Canada Sept. 21, 2006.

School of Engineering and Computer Science Victoria University of Wellington Copyright: Peter Andreae, VUW Image Recognition COMP # 18.

Multiple Instance Learning for Sparse Positive Bags Razvan C. Bunescu Machine Learning Group Department of Computer Sciences University of Texas at Austin.

Image Decomposition, Inpainting, and Impulse Noise Removal by Sparse & Redundant Representations Michael Elad The Computer Science Department The Technion.

Mentor Prof. Amitabha Mukerjee Deepak Pathak Kaustubh Tapi 10346

Learning Photographic Global Tonal Adjustment with a Database of Input / Output Image Pairs.

Convolutional Restricted Boltzmann Machines for Feature Learning Mohammad Norouzi Advisor: Dr. Greg Mori Simon Fraser University 27 Nov

Object Recognizing. Deep Learning Success in 2012 DeepNet and speech processing.

Logan Lebanoff Mentor: Haroon Idrees

Parsing Natural Scenes and Natural Language with Recursive Neural Networks INTERNATIONAL CONFERENCE ON MACHINE LEARNING (ICML 2011) RICHARD SOCHER CLIFF.

Ke (Kevin) Wu1,2, Philip Watters1, Malik Magdon-Ismail1

Hybrid Deep Learning for Reflectance Confocal Microscopy Skin Images

Unsupervised Learning of Video Representations using LSTMs

Deep Learning for Dual-Energy X-Ray

Learning to Compare Image Patches via Convolutional Neural Networks

Semi-Supervised Clustering

Faster R-CNN – Concepts

The Relationship between Deep Learning and Brain Function

Data Mining, Neural Network and Genetic Programming

Computational Thinking, Problem-solving and Programming: General Principals IB Computer Science.

Week 3 (June 6 – June10 , 2016) Summary :

Goodfellow: Chap 1 Introduction

Matt Gormley Lecture 16 October 24, 2016

Oral Presentation 8th Advanced Second semester 2017

Part-Based Room Categorization for Household Service Robots

Hierarchical Deep Convolutional Neural Network

Synthesis of X-ray Projections via Deep Learning

Natural Language Processing of Knee MRI Reports

EXAMPLE 1 Represent relations

Classification of Hand-Written Digits Using Scattering Convolutional Network Dongmian Zou Advisor: Professor Radu Balan.

Deep learning and applications to Natural language processing

Image Inpainting Using Pre-trained Classification CNN

AV Autonomous Vehicles.

CS 698 | Current Topics in Data Science

CS6890 Deep Learning Weizhen Cai

Project Implementation for ITCS4122

Goodfellow: Chap 1 Introduction

State-of-the-art face recognition systems

Bird-species Recognition Using Convolutional Neural Network

Distributed Representation of Words, Sentences and Paragraphs

CSC 578 Neural Networks and Deep Learning

Prepared by: Mahmoud Rafeek Al-Farra

RGB-D Image for Scene Recognition by Jiaqi Guo

Yaniv Romano The Electrical Engineering Department

Introduction of MATRIX CAPSULES WITH EM ROUTING

Object Classification through Deconvolutional Neural Networks

[Figure taken from googleblog

Pattern Recognition & Machine Learning

Forward and Backward Max Pooling

Depth Aware Inpainting for Novel View Synthesis Jayant Thatte

Improving K-SVD Denoising by Post-Processing its Method-Noise

2.1 Review When variables change together, their interaction is called a ___________. When one variable determines the exact value of a second variable,

Department of Computer Science Ben-Gurion University of the Negev

Department of Computer Science Ben-Gurion University of the Negev

Chuan Wang1, Haibin Huang1, Xiaoguang Han2, Jue Wang1

Learning and Memorization

CSC 578 Neural Networks and Deep Learning

Do Better ImageNet Models Transfer Better?

Presentation transcript:

Image Inpainting Using Pre-Trained Classification CNN By - Yaniv Kerzhner & Adar Elad Supervisor - Dr. Yaniv Romano

The process of reconstructing lost parts in a given image The Problem - Image Inpainting The process of reconstructing lost parts in a given image M This project aims to apply convolutional neural networks (CNN) for solving the inpainting problem Our prime goal - Investigate whether a pre-trained CNN classifier (for digits/faces) has memorized the data in order to fill it in Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

CNN – Convolutional Neural Networks Input Output Lena Bengio Abbel Hinton Le-Cun Lin 1% 90% 2% Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

The straight-forward solution Given a set of training pairs (original and corrupted images), train a network in a supervised fashion Neural Network ℒ , Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source. Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

The straight-forward solution Given a set of training pairs (original and corrupted images), train a network in a supervised fashion Nian Cai. et. al. (2015) J. Xie et. al. (2012) D. Pathak et. al. (2016) Raymond A, et. al. (2017) Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source. Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

The Proposed Method Pre-trained CNN car house lizard tree Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source. Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

The Proposed Method Pre-trained CNN car house lizard tree Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source. Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

The Proposed Method Pre-trained CNN car house lizard tree Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source. Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

The Proposed Method Pre-trained CNN car house lizard tree Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source. Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

The Proposed Method Pre-trained CNN car house lizard tree Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source. Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

The Proposed Method The Proposed Method Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

Results – MNIST dataset original Masked Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source. Our Result Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

Results – Yale Extended B dataset original Masked Our Result Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source. Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

90% of the image is missing !! Results – ImageNet dataset 90% of the image is missing !! Masked Our Result Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source. Adar Elad & Yaniv Kerzhner The Computer-Science Department The Technion

Results – Special Cases Hallucination - The specific case of image inpainting where all the data is missing 3 5 8 Illumination - Many face images in the dataset were taken in the dark, the question then arises: could these images be illuminated with the assistant of our algorithm? input It is possible to distinguish that all the details of the original image still remains after the inpainting. Meaning, all we did is to add information to the dark parts. our result Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source.

Time to Conclude Time to Conclude Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source.

Time to Conclude Suggested Interaction Timing: After showing the entire content of this slide Question: Will this algorithm operate well on radar (SAR) images (show such images)? Problem Type: Checkbox Problem with answers YES or NO, and explain Problem Type: Checkbox Problem with answers YES or NO, and explain in a discussion Then show the answer – No, because the model does not match this data source.