A Personal Surround Environment: Projective Display with Correction for Display Surface Geometry and Extreme Lens Distortion Tyler Johnson, Florian Gyarfas,

Slides:

Advertisements

Similar presentations

Point-based Graphics for Estimated Surfaces

Advertisements

Real-Time Projector Tracking on Complex Geometry Using Ordinary Imagery Tyler Johnson and Henry Fuchs University of North Carolina – Chapel Hill ProCams.

CS123 | INTRODUCTION TO COMPUTER GRAPHICS Andries van Dam © 1/16 Deferred Lighting Deferred Lighting – 11/18/2014.

Some problems... Lens distortion  Uncalibrated structure and motion recovery assumes pinhole cameras  Real cameras have real lenses  How can we.

RealityEngine Graphics Kurt Akeley Silicon Graphics Computer Systems.

Cameras and Projectors

Multimedia Specification Design and Production 2012 / Semester 1 / week 6 Lecturer: Dr. Nikos Gazepidis

IITB-Monash Research Academy An Indian-Australian Research Partnership IIT Bombay Projection Defocus Correction using Adaptive Kernel Sampling and Geometric.

Week 7 - Monday.  What did we talk about last time?  Specular shading  Aliasing and antialiasing.

Week 10 - Monday.  What did we talk about last time?  Global illumination  Shadows  Projection shadows  Soft shadows.

I3D Fast Non-Linear Projections using Graphics Hardware Jean-Dominique Gascuel, Nicolas Holzschuch, Gabriel Fournier, Bernard Péroche I3D 2008.

A Distributed Cooperative Framework for Continuous Multi- Projector Pose Estimation IEEE VR March 16, 2009 Tyler Johnson, Greg Welch, Henry Fuchs,

Rasterization and Ray Tracing in Real-Time Applications (Games) Andrew Graff.

Copyright  Philipp Slusallek Cs fall IBR: Model-based Methods Philipp Slusallek.

A Unified Multi-Surface, Multi- Resolution Workspace with Camera-Based Scanning and Projector- Based Illumination Tyler Johnson and Henry Fuchs University.

RANSAC-Assisted Display Model Reconstruction for Projective Display Patrick Quirk, Tyler Johnson, Rick Skarbez, Herman Towles, Florian Gyarfas, Henry Fuchs.

18.1 Si31_2001 SI31 Advanced Computer Graphics AGR Lecture 18 Image-based Rendering Light Maps What We Did Not Cover Learning More...

Advanced Computer Graphics (Spring 2005) COMS 4162, Lecture 21: Image-Based Rendering Ravi Ramamoorthi

Creating Adaptive Views for Group Video Teleconferencing – An Image-Based Approach Creating Adaptive Views for Group Video Teleconferencing – An Image-Based.

1 Combining Approximate Geometry with VDTM – A Hybrid Approach to 3D Video Teleconferencing Celso Kurashima 2, Ruigang Yang 1, Anselmo Lastra 1 1 Department.

(conventional Cartesian reference system)

1 Image-Based Visual Hulls Paper by Wojciech Matusik, Chris Buehler, Ramesh Raskar, Steven J. Gortler and Leonard McMillan [

Real-Time Geometric and Color Calibration for Multi-Projector Displays Christopher Larson, Aditi Majumder Large-Area High Resolution Displays Motivation.

3D Graphics Processor Architecture Victor Moya. PhD Project Research on architecture improvements for future Graphic Processor Units (GPUs). Research.

Flexible Bump Map Capture From Video James A. Paterson and Andrew W. Fitzgibbon University of Oxford Calibration Requirement:

IN4151 Introduction 3D graphics 1 Introduction to 3D computer graphics part 2 Viewing pipeline Multi-processor implementation GPU architecture GPU algorithms.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL An Incremental Weighted Least Squares Approach To Surface Light Fields Greg Coombe Anselmo Lastra.

Pixel Shader Vertex Shader The Real-time Graphics Pipeline Input Assembler Rasterizer Output Merger.

CSCE 641 Computer Graphics: Image-based Modeling Jinxiang Chai.

CSCE 641 Computer Graphics: Image-based Rendering (cont.) Jinxiang Chai.

CSCE 641: Computer Graphics Image-based Rendering Jinxiang Chai.

1 Angel: Interactive Computer Graphics 4E © Addison-Wesley 2005 Models and Architectures Ed Angel Professor of Computer Science, Electrical and Computer.

Basic Principles of Imaging and Photometry Lecture #2 Thanks to Shree Nayar, Ravi Ramamoorthi, Pat Hanrahan.

Filtering Approaches for Real-Time Anti-Aliasing /

Marching Cubes: A High Resolution 3D Surface Construction Algorithm

Technology and Historical Overview. Introduction to 3d Computer Graphics  3D computer graphics is the science, study, and method of projecting a mathematical.

Computer graphics & visualization REYES Render Everything Your Eyes Ever Saw.

1 Image Basics Hao Jiang Computer Science Department Sept. 4, 2014.

Epipolar geometry The fundamental matrix and the tensor

KinectFusion : Real-Time Dense Surface Mapping and Tracking IEEE International Symposium on Mixed and Augmented Reality 2011 Science and Technology Proceedings.

Advanced Computer Graphics March 06, Grading Programming assignments Paper study and reports (flipped classroom) Final project No written exams.

Interactive Time-Dependent Tone Mapping Using Programmable Graphics Hardware Nolan GoodnightGreg HumphreysCliff WoolleyRui Wang University of Virginia.

09/09/03CS679 - Fall Copyright Univ. of Wisconsin Last Time Event management Lag Group assignment has happened, like it or not.

Reconstructing 3D mesh from video image sequences supervisor : Mgr. Martin Samuelčik by Martin Bujňák specifications Master thesis

C O M P U T E R G R A P H I C S Guoying Zhao 1 / 14 C O M P U T E R G R A P H I C S Guoying Zhao 1 / 14 Going-through.

09/11/03CS679 - Fall Copyright Univ. of Wisconsin Last Time Graphics Pipeline Texturing Overview Cubic Environment Mapping.

CSC 461: Lecture 3 1 CSC461 Lecture 3: Models and Architectures  Objectives –Learn the basic design of a graphics system –Introduce pipeline architecture.

1 Introduction to Computer Graphics with WebGL Ed Angel Professor Emeritus of Computer Science Founding Director, Arts, Research, Technology and Science.

1Computer Graphics Lecture 4 - Models and Architectures John Shearer Culture Lab – space 2

1 The Rendering Pipeline. CS788 Topic of HCI 2 Outline  Introduction  The Graphics Rendering Pipeline  Three functional stages  Example  Bottleneck.

AMIPS: The Anywhere, Multipurpose Image Projection System Nicholas Lord Budirijanto Purnomo Paul Alan Roberts Johns Hopkins University Department of Computer.

Tone Mapping on GPUs Cliff Woolley University of Virginia Slides courtesy Nolan Goodnight.

M. Jędrzejewski, K.Marasek, Warsaw ICCVG, Multimedia Chair Computation of room acoustics using programable video hardware Marcin Jędrzejewski.

Review on Graphics Basics. Outline Polygon rendering pipeline Affine transformations Projective transformations Lighting and shading From vertices to.

Lecture 6 Rasterisation, Antialiasing, Texture Mapping,

Graphics Interface 2009 The-Kiet Lu Kok-Lim Low Jianmin Zheng 1.

Real-Time Relief Mapping on Arbitrary Polygonal Surfaces Fabio Policarpo Manuel M. Oliveira Joao L. D. Comba.

1 Angel: Interactive Computer Graphics5E © Addison- Wesley 2009 Image Formation Fundamental imaging notions Fundamental imaging notions Physical basis.

CSCE 641 Computer Graphics: Image-based Rendering (cont.) Jinxiang Chai.

Ray Tracing using Programmable Graphics Hardware

Ray Tracing by GPU Ming Ouhyoung. Outline Introduction Graphics Hardware Streaming Ray Tracing Discussion.

1cs426-winter-2008 Notes. 2 Atop operation  Image 1 “atop” image 2  Assume independence of sub-pixel structure So for each final pixel, a fraction alpha.

1 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 Models and Architectures 靜宜大學資訊工程系蔡奇偉副教授 2012.

1 Interactive Volume Isosurface Rendering Using BT Volumes John Kloetzli Marc Olano Penny Rheingans UMBC.

Applications and Rendering pipeline

The Graphics Rendering Pipeline

Introduction to Computer Graphics with WebGL

Multiple View Geometry for Robotics

Current Research in VR Display Technology

Presentation transcript:

A Personal Surround Environment: Projective Display with Correction for Display Surface Geometry and Extreme Lens Distortion Tyler Johnson, Florian Gyarfas, Rick Skarbez, Herman Towles and Henry Fuchs Department of Computer Science University of North Carolina at Chapel Hill IEEE VR– March 14, 2007

A Personal Surround Environment2 Multi-Projector Display

A Personal Surround Environment3 Wide-FOV Projectors Conventional Projector Fisheye-Lens Projector Conventional vs. fisheye lens projector

A Personal Surround Environment4 Outline Distortion due to Non-Planar Surfaces Lens Distortion Correction for Projectors Results Contributions and Future Work

A Personal Surround Environment5 Non-planar Display Surfaces P

A Personal Surround Environment6 Geometry Compensation via Ray Tracing Display Surface Viewer Projector Projector Image Required Information Viewer location Display surface model Projector calibration

A Personal Surround Environment7 Classic Remapping Algorithm 2-Pass Rendering (Raskar ’98) Pass 1: Render application scene Pass 2: Remap Pass-1 Image Viewer Projector

A Personal Surround Environment8 2-Pass Rendering P Pass-1Pass-2 Display Surface Geometry Pinhole Projection Assumes no lens distortion!

A Personal Surround Environment9 Distortion with Wide-FOV Lenses Input Image Projected Result

A Personal Surround Environment10 Previous Applications of Wide-FOV Projection Elumens VisionStation Hemispherical display screen with fisheye-lens projector for immersive display. Static projector/display surface relationship. Distortion correction based on optical/mechanical design specifications.

A Personal Surround Environment11 Previous Applications of Wide-FOV Projection Konieczny ’05, “A Handheld Flexible Display System” Allows exploration of volume data using a deformable hand-held screen. Dynamic display surface.

A Personal Surround Environment12 Our Goals Our goals were to provide camera-based calibration for wide-FOV projectors geometric correction for arbitrary display surfaces support for tracked viewers

A Personal Surround Environment13 3-Pass Rendering Pass-1Pass-2 Display Surface Geometry Pinhole Projection Pass-3 Lens Distortion Correction

A Personal Surround Environment14 3-Pass Rendering Issues for Wide-FOV Lenses Fisheye lenses are highly non-linear Pinhole approximation in Pass-2 leads to sampling issues during lens distortion correction in Pass-3.

A Personal Surround Environment15 Sampling Artifacts Display Surface P 3-Pass Rendering All Passes at Projector Resolution (1024 x 768)

A Personal Surround Environment16 Pass-3 Sampling Issues Fisheye and pinhole lenses have very different spatial sampling characteristics Leads to sampling issues during lens distortion correction. Fisheye lens (output) Pinhole lens (input) Under-sampled! Over-sampled!

A Personal Surround Environment17 Sampling Artifact Reduction Pass 1-2: 4x Projector Resolution (4096x3072) Pass 3: (1024x768) P Display Surface

A Personal Surround Environment18 A New Approach Remove the pinhole approximation of the wide-FOV lens. Use the actual lens model of the projector. Allows simultaneous compensation for lens distortion and non-planar surface geometry.

A Personal Surround Environment19 Changes to Correction Pipeline Pass-1Pass-2 Display Surface Geometry Pinhole Projection Pass-3 Pinhole Projection Lens Distortion Correction Projector Lens Model Use actual projector lens model to avoid resampling artifacts!

A Personal Surround Environment20 A New Approach Display Surface Viewer Projector Pre-compute where projector rays intersect the display surface! Projector Image

A Personal Surround Environment21 Implementation Pre-compute the projector to display surface (ray) mapping Store this projector-resolution, 2D to 3D map on GPU as a floating-point texture. Static map - independent of viewer location. New Pixel Shader At each projector pixel: Look up vertex on the display surface. Project the vertex into ideal image to obtain output color.

A Personal Surround Environment22 Results All Passes at Projector Resolution (1024x768) Display Surface P

A Personal Surround Environment23 Calibration Capture structured light patterns with camera-pair Calibrate projector Linear calibration followed by non- linear optimization incorporating projector lens model Reconstruct surface geometry RANSAC-based plane fitting technique [Quirk, EDT 06] General reconstruction techniques can also be used

A Personal Surround Environment24 Viewer Tracking Could apply simple 2D-2D mapping Applies only to stationary viewer! Our approach is independent of the viewer location Viewer location can be updated each frame Pass-1 Combined Pass-2 & -3 Remapping 2D - Mapping Only applicable for Stationary Viewer

A Personal Surround Environment25 Heterogeneous 2-Projector Display Also applicable to conventional projectors Hybrid Display with Conventional and Fisheye-lens Projectors Fisheye-lens Projector Display Surface Conventional Projector

A Personal Surround Environment26 Results Video

A Personal Surround Environment27 Contributions Single-pass geometric remapping algorithm Correcting for both extreme lens distortion and display surface geometry Minimizes sampling and FOV issues with fisheye lenses Simple GPU shader program implementation Static 2D:3D mapping stored as 3D floating point texture Demonstrated applicability Personal surround environment with a single fisheye lens projector Conventional or Wide-FOV fisheye AND Heterogeneous displays Applicable to real-time VR applications with user-head tracking

A Personal Surround Environment28 Future Work Methods to further reduce sampling artifacts Mip-mapped Pass-1 textures for geometry optimized sampling. Application of more general lens models [Kannala, IEEE TPAMI, Aug. 06] Real-time, continuous calibration Accommodate projector or display surface changes while the application running. Dynamic vs. static Pass-2 re-map function.

A Personal Surround Environment29 Thank You Funding support: ONR N DARWARS Training Superiority program VIRTE – Virtual Technologies and Environments program Elumenati LLC for loan of fisheye-lens projector