Video: savior, or “more of the same”? 16-721: Learning-Based Methods in Vision A. Efros, CMU, Spring 2009 © A.A. Efros.

Slides:

Advertisements

Similar presentations

Actions in video Monday, April 25 Kristen Grauman UT-Austin.

Advertisements

Patch to the Future: Unsupervised Visual Prediction

Space-time interest points Computational Vision and Active Perception Laboratory (CVAP) Dept of Numerical Analysis and Computer Science KTH (Royal Institute.

Activity Recognition Computer Vision CS 143, Brown James Hays 11/21/11 With slides by Derek Hoiem and Kristen Grauman.

Foreground Modeling The Shape of Things that Came Nathan Jacobs Advisor: Robert Pless Computer Science Washington University in St. Louis.

Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor ：王聖智教授 Student ：周節.

Forward-Backward Correlation for Template-Based Tracking Xiao Wang ECE Dept. Clemson University.

Robust Object Tracking via Sparsity-based Collaborative Model

Adviser ： Ming-Yuan Shieh Student ID ： M Student ： Chung-Chieh Lien VIDEO OBJECT SEGMENTATION AND ITS SALIENT MOTION DETECTION USING ADAPTIVE BACKGROUND.

Local Descriptors for Spatio-Temporal Recognition

3D Computer Vision and Video Computing 3D Vision Topic 4 of Part II Visual Motion CSc I6716 Fall 2011 Cover Image/video credits: Rick Szeliski, MSR Zhigang.

Tracking Objects with Dynamics Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 04/21/15 some slides from Amin Sadeghi, Lana Lazebnik,

Background Subtraction and Matting : Computational Photography Alexei Efros, CMU, Fall 2006 © Yuri Bonder.

Multiple Human Objects Tracking in Crowded Scenes Yao-Te Tsai, Huang-Chia Shih, and Chung-Lin Huang Dept. of EE, NTHU International Conference on Pattern.

Background Subtraction : Rendering and Image Processing Alexei Efros.

Deriving Intrinsic Images from Image Sequences Mohit Gupta Yair Weiss.

Multi-camera Video Surveillance: Detection, Occlusion Handling, Tracking and Event Recognition Oytun Akman.

3D Computer Vision and Video Computing 3D Vision Topic 8 of Part 2 Visual Motion (II) CSC I6716 Spring 2004 Zhigang Zhu, NAC 8/203A

Object Recognition by Parts Object recognition started with line segments. - Roberts recognized objects from line segments and junctions. - This led to.

1 Video Surveillance systems for Traffic Monitoring Simeon Indupalli.

Motion and optical flow Thursday, Nov 20 Many slides adapted from S. Seitz, R. Szeliski, M. Pollefeys, S. Lazebnik.

EE392J Final Project, March 20, Multiple Camera Object Tracking Helmy Eltoukhy and Khaled Salama.

ICCV 2003UC Berkeley Computer Vision Group Recognizing Action at a Distance A.A. Efros, A.C. Berg, G. Mori, J. Malik UC Berkeley.

Recognizing Action at a Distance A.A. Efros, A.C. Berg, G. Mori, J. Malik UC Berkeley.

TP15 - Tracking Computer Vision, FCUP, 2013 Miguel Coimbra Slides by Prof. Kristen Grauman.

Active Vision Key points: Acting to obtain information Eye movements Depth from motion parallax Extracting motion information from a spatio-temporal pattern.

1. Introduction Motion Segmentation The Affine Motion Model Contour Extraction & Shape Estimation Recursive Shape Estimation & Motion Estimation Occlusion.

Vision Surveillance Paul Scovanner.

Periodic Motion Detection via Approximate Sequence Alignment Ivan Laptev*, Serge Belongie**, Patrick Perez* *IRISA/INRIA, Rennes, France **Univ. of California,

Recognizing Human Figures and Actions Greg Mori Simon Fraser University.

Recovering Surface Layout from a Single Image D. Hoiem, A.A. Efros, M. Hebert Robotics Institute, CMU Presenter: Derek Hoiem CS 598, Spring 2009 Jan 29,

Dynamic 3D Scene Analysis from a Moving Vehicle Young Ki Baik (CV Lab.) (Wed)

Introduction EE 520: Image Analysis & Computer Vision.

#MOTION ESTIMATION AND OCCLUSION DETECTION #BLURRED VIDEO WITH LAYERS

Deriving Intrinsic Images from Image Sequences Mohit Gupta 04/21/2006 Advanced Perception Yair Weiss.

Recognizing Action at a Distance Alexei A. Efros, Alexander C. Berg, Greg Mori, Jitendra Malik Computer Science Division, UC Berkeley Presented by Pundik.

Tracking People by Learning Their Appearance Deva Ramanan David A. Forsuth Andrew Zisserman.

Why is computer vision difficult?

Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm Lecture #16.

12/7/10 Looking Back, Moving Forward Computational Photography Derek Hoiem, University of Illinois Photo Credit Lee Cullivan.

Expectation-Maximization (EM) Case Studies

Figure ground segregation in video via averaging and color distribution Introduction to Computational and Biological Vision 2013 Dror Zenati.

Rick Parent - CIS681 Motion Analysis – Human Figure Processing video to extract information of objects Motion tracking Pose reconstruction Motion and subject.

Human Activity Recognition at Mid and Near Range Ram Nevatia University of Southern California Based on work of several collaborators: F. Lv, P. Natarajan,

Lecture 9 Feature Extraction and Motion Estimation Slides by: Michael Black Clark F. Olson Jean Ponce.

CSSE463: Image Recognition Day 29 This week This week Today: Surveillance and finding motion vectors Today: Surveillance and finding motion vectors Tomorrow:

Suspicious Behavior in Outdoor Video Analysis - Challenges & Complexities Air Force Institute of Technology/ROME Air Force Research Lab Unclassified IED.

Today. you will entertain me Enhancing the Throughput of Video Streaming Using Automatic Colorization Sender Automatic Colorization Internet Receiver.

Representation in Vision Derek Hoiem CS 598, Spring 2009 Jan 22, 2009.

Motion Segmentation at Any Speed Shrinivas J. Pundlik Department of Electrical and Computer Engineering, Clemson University, Clemson, SC.

Che-An Wu Background substitution. Background Substitution AlphaMa p Trimap Depth Map Extract the foreground object and put into another background Objective.

Video object segmentation and its salient motion detection using adaptive background generation Kim, T.K.; Im, J.H.; Paik, J.K.; Electronics Letters

Motion and optical flow

CSSE463: Image Recognition Day 29

COMP 9517 Computer Vision Motion 7/21/2018 COMP 9517 S2, 2012.

Tracking Objects with Dynamics

Can computers match human perception?

CSE 577 Image and Video Analysis

Optical Flow Estimation and Segmentation of Moving Dynamic Textures

Presented by Omer Shakil

Vehicle Segmentation and Tracking in the Presence of Occlusions

Vehicle Segmentation and Tracking from a Low-Angle Off-Axis Camera

15-463: Rendering and Image Processing Alexei Efros

CSSE463: Image Recognition Day 29

CSSE463: Image Recognition Day 29

CSSE463: Image Recognition Day 29

CSSE463: Image Recognition Day 29

Week 3 Volodymyr Bobyr.

Presentation transcript:

Video: savior, or “more of the same”? : Learning-Based Methods in Vision A. Efros, CMU, Spring 2009 © A.A. Efros

Why do video? Disambiguation: what is difficult in a single image may become easier in a sequence Improve segmentation spatial reasoning, parallax, occlusions Improve object recognition (e.g. multiview) etc. Understanding temporal phenomena actions, events, activities, object interactions, affordances, etc.

Disambiguation Jackson Pollock Number 21 (detail)

The (alleged) benefits of video Video as space-time volume Object correspondence via tracking “Background subtraction”, time-lapse data 3D from parallax, occlusion reasoning More data (never hurts, right?)

Space-time volume x y t

Sliding XYT windows

3D volume segmentation C. Fowlkes, S. Belongie, F. Chung, J. Malik. "Spectral Grouping Using The Nyström Method", TPAMI 26 (2)TPAMI Erdos # = 1

Problem with space-time volumes Time t is hugely undersampled compared to x,y

Tracking: making t special Optical Flow Feature Tracking (Lucas-Kanade) CHALLENGE: Find me an object tracking paper with more than 15 seconds of resulting video…Tracking eventually fails!

Tracking by repeated recognition

Hybrid approach Ramanan, Forsyth, Zisserman, 2004

Build Model, then Detect Ramanan, Forsyth, Zisserman, 2004

Very impressive result

Stationary Camera assumption What can we do with this?

Background Subtraction - =

A largely unsolved problem… Estimated background Difference Image Thresholded Foreground on blue One video frame

Even when crazy stuff happens…

…averaging can often handle it

Scene-specific Motion Priors Robert Pless er al

Sometimes works for moving cameras Irani and Anandan

Time-lapse data (Webcams) Claude Monet, Haystacks studies

“Smoke” (1996), the “photo album scene”

Removing Shadows (Weiss, 2001) How does one detect (subtract away) shadows?

Averaging Derivatives

Recovering Shadows

Compositing with Shadows

Motion for parallax, occlusions Stein, Hoiem, et alHoiem, Stein, et al Parallax tricky for forward movement

Lots of Data is good! YouTube-style Data … Flickr Data

Temporal Phenomena So, what’s the big deal about motion/video? Probably, temporal phenomena: Actions Activities Functional categories Object interactions Affordances (probably as priors) Etc.