Download presentation
Presentation is loading. Please wait.
1
Video: savior, or “more of the same”? 16-721: Learning-Based Methods in Vision A. Efros, CMU, Spring 2009 © A.A. Efros
2
Why do video? Disambiguation: what is difficult in a single image may become easier in a sequence Improve segmentation spatial reasoning, parallax, occlusions Improve object recognition (e.g. multiview) etc. Understanding temporal phenomena actions, events, activities, object interactions, affordances, etc.
3
Disambiguation Jackson Pollock Number 21 (detail)
4
The (alleged) benefits of video Video as space-time volume Object correspondence via tracking “Background subtraction”, time-lapse data 3D from parallax, occlusion reasoning More data (never hurts, right?)
5
Space-time volume x y t
6
Sliding XYT windows
7
3D volume segmentation C. Fowlkes, S. Belongie, F. Chung, J. Malik. "Spectral Grouping Using The Nyström Method", TPAMI 26 (2)TPAMI Erdos # = 1
8
Problem with space-time volumes Time t is hugely undersampled compared to x,y
9
Tracking: making t special Optical Flow Feature Tracking (Lucas-Kanade) CHALLENGE: Find me an object tracking paper with more than 15 seconds of resulting video…Tracking eventually fails!
10
Tracking by repeated recognition
11
Hybrid approach Ramanan, Forsyth, Zisserman, 2004
12
Build Model, then Detect Ramanan, Forsyth, Zisserman, 2004
13
Very impressive result http://www.ics.uci.edu/~dramanan/videos/kwanPostSmoothGreen_divx.avi
14
Stationary Camera assumption What can we do with this?
15
Background Subtraction - =
16
A largely unsolved problem… Estimated background Difference Image Thresholded Foreground on blue One video frame
17
Even when crazy stuff happens…
18
…averaging can often handle it
19
Scene-specific Motion Priors Robert Pless er al
20
Sometimes works for moving cameras Irani and Anandan
21
Time-lapse data (Webcams) Claude Monet, Haystacks studies
22
“Smoke” (1996), the “photo album scene” http://www.youtube.com/watch?v=MpCpsExvD4Q&feature=channel_page
23
Removing Shadows (Weiss, 2001) How does one detect (subtract away) shadows?
24
Averaging Derivatives
25
Recovering Shadows
26
Compositing with Shadows
27
Motion for parallax, occlusions Stein, Hoiem, et alHoiem, Stein, et al Parallax tricky for forward movement
28
Lots of Data is good! YouTube-style Data … Flickr Data
29
Temporal Phenomena So, what’s the big deal about motion/video? Probably, temporal phenomena: Actions Activities Functional categories Object interactions Affordances (probably as priors) Etc.
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.