Self-Supervised Cross-View Action Synthesis

Slides:

Advertisements

Similar presentations

ETISEO Project Corpus data - Video sequences contents - Silogic provider.

Advertisements

CDA 3100 Recitation Week 10.

The fundamental matrix F

Y A S O O B A L I b o r n o n 1 9 t h F e b r u a r y i n K a n p u r d i s t r i c t o f U t t a r P r a d e s h. H e s t a r t e d s i n g i.

1 Face Tracking in Videos Gaurav Aggarwal, Ashok Veeraraghavan, Rama Chellappa.

Yuping Lin and Gérard Medioni.  Introduction  Method  Register UAV streams to a global reference image ▪ Consecutive UAV image registration ▪ UAV to.

University of Palestine University of Palestine Eng. Wisam Zaqoot Eng. Wisam Zaqoot May 2011 May 2011 Steganalysis ITSS 4201 Internet Insurance and Information.

International Conference on Computer Vision and Graphics, ICCVG ‘2002 Algorithm for Fusion of 3D Scene by Subgraph Isomorphism with Procrustes Analysis.

Math – What is a Function? 1. 2 input output function.

Student Name: Honghao Chen Supervisor: Dr Jimmy Li Co-Supervisor: Dr Sherry Randhawa.

WEEK 1 You have 10 seconds to name…

End of Week 3 + Weekend Work UCF Computer Vision REU 2012 Paul Finkel 6/4/12.

Height Estimation from Egocentric Video- Week 1 Dr. Ali Borji Aisha Urooj Khan Jessie Finocchiaro UCF CRCV REU 2016.

Machine learning & object recognition Cordelia Schmid Jakob Verbeek.

My Research in a Nut-Shell Michael Elad The Computer Science Department The Technion – Israel Institute of technology Haifa 32000, Israel Meeting with.

A Hierarchical Deep Temporal Model for Group Activity Recognition

CPMGT 300 Week 3 Learning Team Planning Process Groups and Developing the Scope Check this A+ tutorial guideline at

CPMGT 300 Week 4 Learning Team Collecting Requirements Check this A+ tutorial guideline at 300/CPMGT-300-Week-4-Learning-Team-Collecting-

Unsupervised Learning of Video Representations using LSTMs

Action-Grounded Push Affordance Bootstrapping of Unknown Objects

Week 3 (June 6 – June10 , 2016) Summary :

Textual Video Prediction Week 2

Query-Focused Video Summarization – Week 1

Evaluating Techniques for Image Classification

Lecture 3: Linear Regression (with One Variable)

Image Restoration using Model-based Tracking

Week 8 REU Nolan Warner.

Compositional Human Pose Regression

Model Functions Input x 6 = Output Input x 3 = Output

Textual Video Prediction

Modeling Motion Blur in Computer – Generated Images

MOTION ESTIMATION AND VIDEO COMPRESSION

Rob Fergus Computer Vision

مادة الدرس : مقدمة في علم الإحصاء

CellNetQL Image Segmentation without Feature Definition

Road Traffic Sign Recognition

Week 7 Nicholas Baker.

Project 7: Modeling Social Network Structures and their Dynamic Evolutions with User- Generated Data from IoT REU Student: Emma Ambrosini Graduate mentors:

Function Notation “f of x” Input = x Output = f(x) = y.

Nolan Warner Week 12 REU.

Depth Aware Inpainting for Novel View Synthesis Jayant Thatte

Video Imagination from a Single Image with Transformation Generation

Optical flow and keypoint tracking

X Y Relation (a set of ordered pairs) x y x y ( , ) x y Mapping

CRCV REU UCF Summer 2019 Arisa Kitagishi.

Nolan Warner Week 11 REU.

Multi-UAV to UAV Tracking

Deep screen image crop and enhance

Weak-supervision based Multi-Object Tracking

CRCV REU 2019 Kara Schatz.

Appearance Transformer (AT)

Week 3 Volodymyr Bobyr.

Self-Supervised Cross-View Action Synthesis

CRCV REU 2019 Week 5.

SFNet: Learning Object-aware Semantic Correspondence

Deep screen image crop and enhance

Self-Supervised Cross-View Action Synthesis

End-to-End Speech-Driven Facial Animation with Temporal GANs

Week 7 Presentation Ngoc Ta Aidean Sharghi

Self-Supervised Cross-View Action Synthesis

Project Presentation – Week 6

Self-Supervised Cross-View Action Synthesis

Week 6: Moving Target Detection Using Infrared Sensors

Week 5 Cecilia La Place.

CRCV REU 2019 Week 4.

REU Program 2019 Week 5 Alex Ruiz Jyoti Kini.

REU Program 2019 Week 6 Alex Ruiz Jyoti Kini.

Week 4: Moving Target Detection Using Infrared Sensors

Presentation transcript:

Self-Supervised Cross-View Action Synthesis Kara Schatz Advisor: Dr. Yogesh Rawat UCF CRCV – REU, Summer 2019

Synthesize a video from an unseen view. Project Goal Synthesize a video from an unseen view. Given: video of the same scene from a different viewpoint single image from the desired viewpoint

Last Week’s Results . . . . . . FRAME 1 FRAME 2 FRAME 15 FRAME 16 Ground Truth: . . . Output:

Approach Action Rep. Action Rep.

Approach Action Rep. Action Rep. This week, we went back to this original approach, but

Approach Action Rep. Action Rep. This week, we went back to this original approach, but

Output without Consistency Loss NTU: 66 epochs Panoptic: 72 epochs Input: Output: Ground Truth: Input: Output: Ground Truth: Frame 1: Frame 16:

Output without Consistency Loss NTU: 66 epochs Panoptic: 72 epochs Input: Output: Ground Truth: Input: Output: Ground Truth: Frame 1: Frame 16:

Approach Trans- formation Trans-formation Estimated Key-points Action viewpoint Trans- formation Estimated Keypoints Key-points Action Rep. Trans-formation Estimated Keypoints Key-points Action Rep. viewpoint

Output with Keypoints NTU: 98 epochs Panoptic: 108 epochs Input: Ground Truth: Input: Output: Ground Truth: Frame 1: Frame 16:

Output with Keypoints NTU: 98 epochs Panoptic: 108 epochs Input: Ground Truth: Input: Output: Ground Truth: Frame 1: Frame 16:

Keypoints Keypoint Transformed View 1: View 2:

Next Steps

Next Steps Trans- formation Trans-formation Estimated Key-points viewpoint Trans- formation Estimated Keypoints Key-points Action Rep. Trans-formation Estimated Keypoints Key-points Action Rep. viewpoint

Next Steps Trans- formation Consistency losses Trans-formation viewpoint Trans- formation Estimated Keypoints Key-points Action Rep. Consistency losses Trans-formation Estimated Keypoints Key-points Action Rep. viewpoint

Next Steps Trans- formation Trans-formation Action Key-points viewpoint Trans- formation Action Rep. Est. Key-points Action Rep. Action Rep. Trans-formation Action Rep. Est. Key-points viewpoint

Next Steps Predict Keypoints Trans- formation Predict Keypoints viewpoint Predict Keypoints Trans- formation Estimated Keypoints Action Rep. Est. Key-points Key-points Predict Keypoints Action Rep. Key-points Predict Keypoints Key-points Action Rep. Trans-formation Predict Keypoints Estimated Keypoints Action Rep. Est. Key-points Key-points viewpoint

Next Steps Predict Keypoints Trans- formation Predict Keypoints viewpoint Predict Keypoints Trans- formation Estimated Keypoints Action Rep. Est. Key-points Key-points Predict Keypoints Action Rep. Key-points Predict Keypoints Key-points Action Rep. Trans-formation Predict Keypoints Estimated Keypoints Action Rep. Est. Key-points Key-points viewpoint

Next Steps Predict Keypoints Trans- formation Predict Keypoints viewpoint Predict Keypoints Trans- formation Estimated Keypoints Action Rep. Est. Key-points Key-points Predict Keypoints Action Rep. Key-points Predict Keypoints Key-points Action Rep. Trans-formation Predict Keypoints Estimated Keypoints Action Rep. Est. Key-points Key-points viewpoint