Michal Irani Dept of Computer Science and Applied Math Weizmann Institute of Science Rehovot, ISRAEL Spatio-Temporal Analysis and Manipulation of Visual.

Slides:

Advertisements

Similar presentations

Games, Movies and Virtual Worlds – An Introduction to Computer Graphics Ayellet Tal Department of Electrical Engineering Technion.

Advertisements

INTERNATIONAL CONFERENCE ON TELECOMMUNICATIONS, ICT '09. TAREK OUNI WALID AYEDI MOHAMED ABID NATIONAL ENGINEERING SCHOOL OF SFAX New Low Complexity.

Sequence-to-Sequence Alignment and Applications. Video > Collection of image frames.

Patch to the Future: Unsupervised Visual Prediction

Activity Recognition Aneeq Zia. Agenda What is activity recognition Typical methods used for action recognition “Evaluation of local spatio-temporal features.

Vision Based Control Motion Matt Baker Kevin VanDyke.

Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor ：王聖智教授 Student ：周節.

Robust Object Tracking via Sparsity-based Collaborative Model

2D Human Pose Estimation in TV Shows Vittorio Ferrari Manuel Marin Andrew Zisserman Dagstuhl Seminar July 2008.

Nonchronological Video Synopsis and Indexing TPAMI 2008 Yael Pritch, Alex Rav-Acha, and Shmuel Peleg, Member, IEEE 1.

Local Descriptors for Spatio-Temporal Recognition

X From Video - Seminar By Randa Khayr Eli Shechtman, Yaron Caspi & Michal Irani.

Motion Estimation I What affects the induced image motion? Camera motion Object motion Scene structure.

On-Demand Sharing of a High-Resolution Panorama Video from Networked Robotic Cameras Supported in part by CPSC 643 Dezhen Song Texas A&M University.

ADVISE: Advanced Digital Video Information Segmentation Engine

Research Update and Future Work Directions – Jan 18, 2006 – Ognjen Arandjelović Roberto Cipolla.

Direct Methods for Visual Scene Reconstruction Paper by Richard Szeliski & Sing Bing Kang Presented by Kristin Branson November 7, 2002.

T.Sharon 1 Internet Resources Discovery (IRD) Video IR.

Static Image Mosaicing

Synchronization and Calibration of Camera Networks from Silhouettes Sudipta N. Sinha Marc Pollefeys University of North Carolina at Chapel Hill, USA.

Computer Vision Optical Flow Marc Pollefeys COMP 256 Some slides and illustrations from L. Van Gool, T. Darell, B. Horn, Y. Weiss, P. Anandan, M. Black,

Motion Estimation I What affects the induced image motion? Camera motion Object motion Scene structure This lesson – unconstrained flow-fields Next lesson.

What is “Image Processing and Computer Vision”?

A Probabilistic Framework for Video Representation Arnaldo Mayer, Hayit Greenspan Dept. of Biomedical Engineering Faculty of Engineering Tel-Aviv University,

Symbolic Description and Visual Querying of Image Sequences Using Spatio-Temporal Logic Del Bimbo, E. Vicario and D. Zingoni IEEE Transactions on Knowledge.

Convergence of vision and graphics Jitendra Malik University of California at Berkeley Jitendra Malik University of California at Berkeley.

Motion Estimation I What affects the induced image motion? Camera motion Object motion Scene structure.

Projects in the Intelligent User Interfaces Group Frank Shipman Associate Director, Center for the Study of Digital Libraries.

Creating and Exploring a Large Photorealistic Virtual Space INRIA / CSAIL / Adobe First IEEE Workshop on Internet Vision, associated with CVPR 2008.

3D/Multview Video. Outline Introduction 3D Perception and HVS 3D Displays 3D Video Representation Compression.

A Brief Overview of Computer Vision Jinxiang Chai.

IRISA / INRIA Rennes Computational Vision and Active Perception Laboratory (CVAP) KTH (Royal Institute of Technology)

1 Mean shift and feature selection ECE 738 course project Zhaozheng Yin Spring 2005 Note: Figures and ideas are copyrighted by original authors.

Marcin Marszałek, Ivan Laptev, Cordelia Schmid Computer Vision and Pattern Recognition, CVPR Actions in Context.

Periodic Motion Detection via Approximate Sequence Alignment Ivan Laptev*, Serge Belongie**, Patrick Perez* *IRISA/INRIA, Rennes, France **Univ. of California,

Object Stereo- Joint Stereo Matching and Object Segmentation Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on Michael Bleyer Vienna.

Dynamic 3D Scene Analysis from a Moving Vehicle Young Ki Baik (CV Lab.) (Wed)

Combined Central and Subspace Clustering for Computer Vision Applications Le Lu 1 René Vidal 2 1 Computer Science Department, Johns Hopkins University,

University of Toronto Aug. 11, 2004 Learning the “Epitome” of a Video Sequence Information Processing Workshop 2004 Vincent Cheung Probabilistic and Statistical.

Spatio-temporal constraints for recognizing 3D objects in videos Nicoletta Noceti Università degli Studi di Genova.

CS 8690: Computer Vision Ye Duan. CS8690 Computer Vision University of Missouri at Columbia Instructor Ye Duan (209 Engr West)

Dynamosaicing Dynamosaicing Mosaicing of Dynamic Scenes (Fri) Young Ki Baik Computer Vision Lab Seoul National University.

Coordinate-Invariant Methods For Motion Analysis and Synthesis Jehee Lee Dept. Of Electric Engineering and Computer Science Korea Advanced Institute of.

Uses of Motion 3D shape reconstruction Segment objects based on motion cues Recognize events and activities Improve video quality Track objects Correct.

Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm Lecture #16.

Advances in digital image compression techniques Guojun Lu, Computer Communications, Vol. 16, No. 4, Apr, 1993, pp

Epitomic Location Recognition A generative approach for location recognition K. Ni, A. Kannan, A. Criminisi and J. Winn In proc. CVPR Anchorage,

Efficient Visual Object Tracking with Online Nearest Neighbor Classifier Many slides adapt from Steve Gu.

CAMEO: Face Recognition Year 1 Progress and Year 2 Goals Fernando de la Torre, Carlos Vallespi, Takeo Kanade.

Student Name: Honghao Chen Supervisor: Dr Jimmy Li Co-Supervisor: Dr Sherry Randhawa.

Large Scale Time-Varying Data Visualization Han-Wei Shen Department of Computer and Information Science The Ohio State University.

 Present by 陳群元.  Introduction  Previous work  Predicting motion patterns  Spatio-temporal transition distribution  Discerning pedestrians  Experimental.

Tracking-dependent and interactive video projection (Big Brother project)

Motion Estimation I What affects the induced image motion?

Semantic Extraction and Semantics-Based Annotation and Retrieval for Video Databases Authors: Yan Liu & Fei Li Department of Computer Science Columbia.

Course14 Dynamic Vision. Biological vision can cope with changing world Moving and changing objects Change illumination Change View-point.

ENTERFACE 08 Project 9 “ Tracking-dependent and interactive video projection ” Mid-term presentation August 19th, 2008.

Visual Information Retrieval

Automatic Video Shot Detection from MPEG Bit Stream

Interactive Offline Tracking for Color Objects

Intrinsic images and shape refinement

Computer Vision, Robotics, Machine Learning and Control Lab

Optical Flow Estimation and Segmentation of Moving Dynamic Textures

Generalized Principal Component Analysis CVPR 2008

Presented by Omer Shakil

The Brightness Constraint

Iterative Optimization

Kan Liu, Bingpeng Ma, Wei Zhang, Rui Huang

PRAKASH CHOCKALINGAM, NALIN PRADEEP, AND STAN BIRCHFIELD

Presentation transcript:

Michal Irani Dept of Computer Science and Applied Math Weizmann Institute of Science Rehovot, ISRAEL Spatio-Temporal Analysis and Manipulation of Visual Information

3. Geometric information 2. Extended temporal coverage Space-Time Visual Information VideoCollection of images 1. Extended spatial coverage

From Images to Space-Time Scenes Talk Outline: Single video sequence Multiple video sequences Multiple non-overlapping video sequences Efficient browsing, search, manipulation, enhancement, synthesis, editing, compression, and much much more... Space-time data redundancy RepresentationAnalysis Use it…

Video Browsing, Indexing, and Manipulation [work with P. Anandan, S. Hsu, T. Hassner] Redundant & implicit frame-based representation Compact & explicit scene-based representation 1. Extended spatial info 2. Extended temporal info 3. Geometric info

Video Browsing, Indexing, and Manipulation [work with P. Anandan, S. Hsu, T. Hassner] INTERACTIVE DEMO

Sequence 1 Sequence 2 Frame 1 Frame 2 Frame 3 Frame n Frame 1 Frame 2 Frame 3 Frame n Sequence-to-Sequence Alignment [work with Yaron Caspi - CVPR’00]

Sequence 1 Sequence 2 Frame 1 Frame 2 Frame 3 Frame n Frame 1 Frame 2 Frame 3 Frame n (a) Given corresponding frames. (b) Find spatial correspondences. (x,y) (x’,y’) Image-to-image alignment No dedicated hardware Should use all spatio-temporal information in video sequences ==> obtain correspondences in space and in time Image-to-Image Alignment Not enough spatial information

Not enough info for alignment in individual frames Image 1 Image 2 Information in Images:

Information in Video: Alignment uniquely defined Information cues for alignment: Appearance info Dynamic info within frames between frames Moving objects Non rigid motion Varying illumination

Spatio-Temporal Alignment SSD Minimization: Gauss-Newton (coarse-to-fine) iterations:

Sequence 1Sequence 2 Before AlignmentAfter Alignment

Sequence 1Sequence 2 Before AlignmentAfter Alignment

Sequence 1Sequence 2 Before AlignmentAfter Alignment Illumination changes:

Sequence 1Sequence 2 Before AlignmentAfter Alignment

Sequence 1Sequence 2 Before Alignment After Alignment

Alignment of Non-Overlapping Sequences Coherent appearance (Image-to-Image Alignment) Sequence-to-Sequence Alignment: Alignment in time and in space Coherent camera behavior Coherent scene dynamics (Seq-to-Seq Alignment) [work with Yaron Caspi - ICCV’01]

H=? Problem formulation H H Input: Output: and such that Sequence 1Sequence 2

Sequence 1: Sequence 2: Spatio-Temporal Alignment Combined Sequence:

Sequence 1: Sequence 2: Application: Wide-Screen Movies Wide- screen movie:

Fused Sequence: Visible light (video): Infra-Red: Application: Multi-Sensor Alignment

Zoomed-in Sequence: Zoomed-out Sequence: Application: Detect Zoomed Region

Wide Field-of-View Narrow Field-of-View Application: Spotting Hooligans

THE END Copyright, 1996 © Dale Carnegie & Associates, Inc. Summary Forget image frames Video >> collection of images Use all spatio-temporal info for representation, analysis, and exploitation.