City Forensics: Using Visual Elements to Predict Non-Visual City Attributes Sean M. Arietta, Alexei A. Efros, Ravi Ramamoorthi, Maneesh Agrawala Presented.

Slides:

Advertisements

Similar presentations

ECG Signal processing (2)

Advertisements

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Object recognition and scene “understanding”

Prachi Saraph, Mark Last, and Abraham Kandel. Introduction Black-Box Testing Apply an Input Observe the corresponding output Compare Observed output with.

Pattern Recognition and Machine Learning

An Introduction of Support Vector Machine

Support Vector Machines

Machine learning continued Image source:

Patch to the Future: Unsupervised Visual Prediction

Automatic in vivo Microscopy Video Mining for Leukocytes * Chengcui Zhang, Wei-Bang Chen, Lin Yang, Xin Chen, John K. Johnstone.

Discriminative and generative methods for bags of features

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

1 Learning to Detect Objects in Images via a Sparse, Part-Based Representation S. Agarwal, A. Awan and D. Roth IEEE Transactions on Pattern Analysis and.

I welcome you all to this presentation On: Neural Network Applications Systems Engineering Dept. KFUPM Imran Nadeem & Naveed R. Butt &

1 Ensembles of Nearest Neighbor Forecasts Dragomir Yankov, Eamonn Keogh Dept. of Computer Science & Eng. University of California Riverside Dennis DeCoste.

R-CNN By Zhang Liliang.

Face Processing System Presented by: Harvest Jang Group meeting Fall 2002.

From R-CNN to Fast R-CNN

Style-aware Mid-level Representation for Discovering Visual Connections in Space and Time Yong Jae Lee, Alexei A. Efros, and Martial Hebert Carnegie Mellon.

Machine learning Image source:

Classification III Tamara Berg CS Artificial Intelligence Many slides throughout the course adapted from Svetlana Lazebnik, Dan Klein, Stuart Russell,

Identifying Computer Graphics Using HSV Model And Statistical Moments Of Characteristic Functions Xiao Cai, Yuewen Wang.

Richard Socher Cliff Chiung-Yu Lin Andrew Y. Ng Christopher D. Manning

End-to-End Text Recognition with Convolutional Neural Networks

Jun-Won Suh Intelligent Electronic Systems Human and Systems Engineering Department of Electrical and Computer Engineering Speaker Verification System.

Classifiers Given a feature representation for images, how do we learn a model for distinguishing features from different classes? Zebra Non-zebra Decision.

BAGGING ALGORITHM, ONLINE BOOSTING AND VISION Se – Hoon Park.

CSSE463: Image Recognition Day 11 Lab 4 (shape) tomorrow: feel free to start in advance Lab 4 (shape) tomorrow: feel free to start in advance Test Monday.

Gang WangDerek HoiemDavid Forsyth. INTRODUCTION APROACH (implement detail) EXPERIMENTS CONCLUSION.

VIP: Finding Important People in Images Clint Solomon Mathialagan Andrew C. Gallagher Dhruv Batra CVPR

Deep Convolutional Nets

Learning Features and Parts for Fine-Grained Recognition Authors: Jonathan Krause, Timnit Gebru, Jia Deng, Li-Jia Li, Li Fei-Fei ICPR, 2014 Presented by:

Learning video saliency from human gaze using candidate selection CVPR2013 Poster.

Convolutional Restricted Boltzmann Machines for Feature Learning Mohammad Norouzi Advisor: Dr. Greg Mori Simon Fraser University 27 Nov

Final Report (30% final score) Bin Liu, PhD, Associate Professor.

Unsupervised Visual Representation Learning by Context Prediction

Rich feature hierarchies for accurate object detection and semantic segmentation 2014 IEEE Conference on Computer Vision and Pattern Recognition Ross Girshick,

Spatial Localization and Detection

Fast Human Detection in Crowded Scenes by Contour Integration and Local Shape Estimation Csaba Beleznai, Horst Bischof Computer Vision and Pattern Recognition,

Neural networks (2) Reminder Avoiding overfitting Deep neural network Brief summary of supervised learning methods.

Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition arXiv: v4 [cs.CV(CVPR)] 23 Apr 2015 Kaiming He, Xiangyu Zhang, Shaoqing.

Parsing Natural Scenes and Natural Language with Recursive Neural Networks INTERNATIONAL CONFERENCE ON MACHINE LEARNING (ICML 2011) RICHARD SOCHER CLIFF.

Facial Smile Detection Based on Deep Learning Features Authors: Kaihao Zhang, Yongzhen Huang, Hong Wu and Liang Wang Center for Research on Intelligent.

High resolution product by SVM. L’Aquila experience and prospects for the validation site R. Anniballe DIET- Sapienza University of Rome.

Neural networks and support vector machines

Data Mining, Machine Learning, Data Analysis, etc. scikit-learn

Deep Learning for Dual-Energy X-Ray

Summary of “Efficient Deep Learning for Stereo Matching”

Article Review Todd Hricik.

Palm Oil Plantation Area Clusterization for Monitoring

R-CNN region By Ilia Iofedov 11/11/2018 BGU, DNN course 2016.

Dynamic Routing Using Inter Capsule Routing Protocol Between Capsules

Cheng-Ming Huang, Wen-Hung Liao Department of Computer Science

A New Approach to Track Multiple Vehicles With the Combination of Robust Detection and Two Classifiers Weidong Min , Mengdan Fan, Xiaoguang Guo, and Qing.

Bird-species Recognition Using Convolutional Neural Network

Computer Vision James Hays

3. Applications to Speaker Verification

Convolutional Neural Networks for Visual Tracking

Hyperparameters, bias-variance tradeoff, validation

Object Detection + Deep Learning

network of simple neuron-like computing elements

Data Mining, Machine Learning, Data Analysis, etc. scikit-learn

RCNN, Fast-RCNN, Faster-RCNN

Data Mining, Machine Learning, Data Analysis, etc. scikit-learn

CSCE833 Machine Learning Lecture 9 Linear Discriminant Analysis

边缘检测年度进展概述 Ming-Ming Cheng Media Computing Lab, Nankai University

Linear Discrimination

Visual Manipulation Relationship Network for Autonomous Robotics

An introduction to Machine Learning (ML)

Presentation transcript:

City Forensics: Using Visual Elements to Predict Non-Visual City Attributes Sean M. Arietta, Alexei A. Efros, Ravi Ramamoorthi, Maneesh Agrawala Presented by: Manu Agarwal

Outline Introduction Related Work Methodology proposed in the paper Results Applications Discussion

One of these is from Paris This is Paris Clap if… Slide credits: Doersch, Singh

Clap if… Slide credits: Doersch, Singh

Modeling predictive relationships Higher greenery City appearance Cases of depression Higher housing prices Broken Windows Theory! Broken glass, graffiti, trashIncreased crime rate

Violent crime rate High housing prices

Related Work Doersch et al. – Method for identifying visual elements of a city that differentiate it from other cities – Binary classification Koller et al., Srinivasan et al. – Use video to track crowds for detecting flow patterns – Availability of such videos is limited

Problem Input: a set of measured (location, attribute- value) pairs and a set of (location, street-side panorama) pairs Output: Predictor that can estimate the value of a non-visual city attribute based on visual appearance

Methodology Spatially interpolate the input (location, attribute-value) data Build a bank of SVMs Build an attribute predictor from the resulting bank of SVMs using SVR

Interpolating Non-Visual City Attribute Values Use the radial basis function (RBF) r is the Euclidean distance between locations Authors use ɛ=2

Methodology Spatially interpolate the input (location, attribute-value) data Build a bank of SVMs Build an attribute predictor from the resulting bank of SVMs using SVR

Constructing the Visual Element Detectors 100k200k300k400k500k600k700k800k900k1M1.1M PositiveNegative Negative set Positive set

Constructing the Visual Element Detectors

Extract the set of image features in each panorama projection HOG+color features

Constructing the Visual Element Detectors Compute 100 nearest neighbors of features sampled randomly from the positive set PatchMatches

Constructing the Visual Element Detectors Compute an SVM for each nearest neighbor set Keep the top 100 SVMs

Methodology Spatially interpolate the input (location, attribute-value) data Build a bank of SVMs Build an attribute predictor from the resulting bank of SVMs using SVR

Computing the Predictor Features SVM scores

Computing the Predictor Retain the top 3 detection scores for each of the 100 SVMs

Computing the Predictor Estimate parameters w and b such that the following loss is minimized ɛ controls the magnitude of error that we can tolerate

Results: Analysis of Prediction Accuracy

Results: Prediction Maps

Results: Deep Convolutional Neural Network Features

Applications: Defining Visual Boundaries of Neighborhoods

Applications: Attribute-Sensitive Wayfinding

Applications: Validating Visual Elements for Prediction

Discussion Presence of other dominant factors controlling non-visual attributes Retain top 5 SVM detection scores Image patch level features vs whole image features Combining predictors from two or more cities Experiment with smaller cities Use fc7 features or finetune CNN

Thank You!