Image Quilting for Texture Synthesis and Transfer Alexei A. Efros (UC Berkeley) William T. Freeman (MERL) Siggraph01 ’

Slides:

Advertisements

Similar presentations

Lapped textures Emil Praun Adam Finkelstein Hugues Hoppe

Advertisements

A Two-Step Approach to Hallucinating Faces: Global Parametric Model and Local Nonparametric Model Ce Liu Heung-Yeung Shum Chang Shui Zhang CVPR 2001.

Filling Algorithms Pixelwise MRFsChaos Mosaics Patch segments are pasted, overlapping, across the image. Then either: Ambiguities are removed by smoothing.

Image Quilting and Apples

Andrew Nealen and Marc Alexa, Discrete Geometric Modeling Group, TU Darmstadt, 2004 Fast and High Quality Overlap Repair for Patch-Based Texture Synthesis.

Super-Resolution Texturing for Online Virtual Globes

Data-driven methods: Texture (Sz 10.5) Cs129 Computational Photography James Hays, Brown, Spring 2011 Many slides from Alexei Efros.

Texture. Edge detectors find differences in overall intensity. Average intensity is only simplest difference. many slides from David Jacobs.

More details on presentations Aim to speak for ~50 min (after 15 min review, leaving 10 min for discussions) Try to plan discussion topics It’s fine to.

Lapped Textures Emil Praun and Adam Finkelstien (Princeton University) Huges Hoppe (Microsoft Research) SIGGRAPH 2000 Presented by Anteneh.

Improved Image Quilting Jeremy Long David Mould. Introduction   Goal: improve “ minimum error boundary cut ”

Maryia Kazakevich “Texture Synthesis by Patch-Based Sampling” Texture Synthesis by Patch-Based Sampling Real-Time Texture Synthesis By Patch-Based Sampling,

Lapped Textures Emil Praun Adam Finkelstein Hugues Hoppe Emil Praun Adam Finkelstein Hugues Hoppe Princeton University Microsoft Research Princeton University.

Announcements Project 4 questions? Guest lectures Thursday: Richard Ladner “tactile graphics” Next Tuesday: Jenny Yuen and Jeff Bigham.

Image Quilting for Texture Synthesis & Transfer Alexei Efros (UC Berkeley) Bill Freeman (MERL) +=

Blending and Compositing : Rendering and Image Processing Alexei Efros.

Texture Synthesis by Non-parametric Sampling / Image Quilting for Texture Synthesis & Transfer by Efros and Leung / Efros and Freeman ICCV ’99 / SIGGRAPH.

Overview of Texture Synthesis Ganesh Ramanarayanan Cornell Graphics Seminar.

Fast Texture Synthesis using Tree-structured Vector Quantization Li-Yi Wei Marc Levoy Computer Graphics Group Stanford University.

Image Quilting for Texture Synthesis and Transfer Alexei A. Efros1,2 William T. Freeman2.

Image Pyramids and Blending

2010/5/171 Overview of graph cuts. 2010/5/172 Outline Introduction S-t Graph cuts Extension to multi-label problems Compare simulated annealing and alpha-

A Crash Course on Texturing Li-Yi Wei 2005 at Tsinghua University, Beijing.

TEXTURE SYNTHESIS PEI YEAN LEE. What is texture? Images containing repeating patterns Local & stationary.

Announcements For future problems sets: matlab code by 11am, due date (same as deadline to hand in hardcopy). Today’s reading: Chapter 9, except.

Simultaneous Structure and Texture Image Inpainting by: Bertalmio, Sapiro, Vese, Osher Presented by: Shane Brennan June 7, 2007 EE 264 – Spring 2007.

Texture Reading: Chapter 9 (skip 9.4) Key issue: How do we represent texture? Topics: –Texture segmentation –Texture-based matching –Texture synthesis.

Texture Synthesis from Multiple Sources Li-Yi Wei Stanford University (was) NVIDIA Corporation (now)

Texture Recognition and Synthesis A Non-parametric Multi-Scale Statistical Model by De Bonet & Viola Artificial Intelligence Lab MIT Presentation by Pooja.

Andrew Nealen and Marc Alexa, Discrete Geometric Modeling Group, TU Darmstadt, 2003 Hybrid Texture Synthesis Andrew Nealen Marc Alexa Discrete Geometric.

Near-Regular Texture Analysis and Manipulation Written by: Yanxi Liu Yanxi Liu Wen-Chieh Lin Wen-Chieh Lin James Hays James Hays Presented by: Alex Hadas.

Texture Synthesis over Arbitrary Manifold Surfaces Li-Yi Wei Marc Levoy Computer Graphics Group Stanford University.

Texture perception Lavanya Sharan February 23rd, 2011.

Image Pyramids and Blending : Computational Photography Alexei Efros, CMU, Fall 2005 © Kenneth Kwan.

Compact Windows for Visual Correspondence via Minimum Ratio Cycle Algorithm Olga Veksler NEC Labs America.

Graphcut Texture: Image and Video Synthesis Using Graph Cuts

Image Pyramids and Blending

Texture Optimization for Example-based Synthesis

Computer Vision Spring ,-685 Instructor: S. Narasimhan WH 5409 T-R 10:30am – 11:50am Lecture #15.

Texture Synthesis by Non-parametric Sampling Alexei Efros and Thomas Leung UC Berkeley.

CAP5415: Computer Vision Lecture 4: Image Pyramids, Image Statistics, Denoising Fall 2006.

Volumetric Illustration: Designing 3D Models with Internal Textures Shigeru Owada Frank Nielsen Makoto Okabe Takeo Igarashi The University of Tokyo Sony.

Texture Optimization for Example-based Synthesis Vivek Kwatra Irfan Essa Aaron Bobick Nipun Kwatra.

Computer Vision, Robert Pless

Advanced Computer Graphics (Fall 2010) CS 283, Lecture 26 Texture Synthesis Ravi Ramamoorthi Slides, lecture.

Synthesis of Compact Textures for real-time Terrain Rendering Nader Salman 22 juin 2007 Encadrant : Sylvain Lefebvre.

Scene Completion Using Millions of Photographs James Hays, Alexei A. Efros Carnegie Mellon University ACM SIGGRAPH 2007.

TextureAmendment Reducing Texture Distortion in Constrained Parameterizations Yu-Wing TaiNational University of Singapore Michael S. BrownNational University.

Towards Real-Time Texture Synthesis With the Jump Map Steve Zelinka Michael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics.

2D Texture Synthesis Instructor: Yizhou Yu. Texture synthesis Goal: increase texture resolution yet keep local texture variation.

Graphcut Textures Image and Video Synthesis Using Graph Cuts

Two Patch-based Algorithms for By-example Texture Synthesis Bruno Galerne MAP5, Université Paris Descartes 1 Master 2 Traitement.

MIT AI Lab / LIDS Laboatory for Information and Decision Systems & Artificial Intelligence Laboratory Massachusetts Institute of Technology A Unified Multiresolution.

The University of Ontario Yuri Boykov Research Interests n Computer Vision n Medical Image Analysis n Graphics Combinatorial optimization algorithms Geometric,

Geometry Synthesis Ares Lagae Olivier Dumont Philip Dutré Department of Computer Science Katholieke Universiteit Leuven 10 August, 2004.

Texture Synthesis by Image Quilting CS766 Class Project Fall 2004 Eric Robinson.

Texture Analysis and Synthesis. Texture Texture: pattern that “looks the same” at all locationsTexture: pattern that “looks the same” at all locations.

Hebrew University Image Processing Exercise Class 8 Panoramas – Stitching and Blending Min-Cut Stitching Many slides from Alexei Efros.

Two Patch-based Algorithms for By-example Texture Synthesis

Graphcut Textures:Image and Video Synthesis Using Graph Cuts

Announcements Project 4 out today help session at the end of class.

Texture Synthesis by Non-parametric Sampling

Announcements Final Project 3 artifacts Evals

By: Kevin Yu Ph.D. in Computer Engineering

Texture Quality Extensions to Image Quilting

Fast Pattern Simulation Using Multi‐Scale Search

Image Quilting for Texture Synthesis & Transfer

Announcements Guest lecture next Tuesday

Outline Texture modeling - continued Julesz ensemble.

Presentation transcript:

Image Quilting for Texture Synthesis and Transfer Alexei A. Efros (UC Berkeley) William T. Freeman (MERL) Siggraph01 ’

About author? Alexei A. Efros Assistant Professor Computer Science Department & The Robotics Institute School of Computer Science Carnegie Mellon University From St. Petersburg, Russia Got PhD from UC Berkeley in 2003 Then a year as a Visiting Research Fellow in Visual Geometry Group of Oxford Joined in CSD and RI in autumn of 2004 Computer graphics & computer vision

About author? William T. Freeman Professor of Electrical Engineering & Computer Science at the Artificial Intelligence Laboratory at MIT(September, 2001) Received a BS in physics and MS in electrical engineering from Stanford (1979), and an MS in applied physics from Cornell(1981) Got his PhD in 1992 from the MIT Worked at Mitsubishi Electric Research Labs (1992 – 2001, Cambridge) Computer vision

Quilting? Transfer? + =

Image vs. Texture

Example-based Texture Synthesis Input Example

The Goal of Texture Synthesis True (infinite) texture SYNTHESIS generated image input image Same in perceptual sense

The Challenge Texture analysis: how to capture the essence of texture? Need to model the whole spectrum: from repeated to stochastic texture RepeatedStochasticBoth?

Related Work Local region-growing method -Pixel-based -Patch-based-Patch-based Global optimization-based method Heeger and Bergen sig95,Pyramid-based texture synthesis Paget and Longstaff IEEE Tran… 98,Texture synthesis via a noncausal nonparametric multiscale markov random field ….. Physical Simulation et al

Input texture B1B2 Random placement of blocks block B1 B2 Neighboring blocks constrained by overlap B1B2 Minimal error boundary cut

min. error boundary Minimum Error Boundary Cut overlapping blocksvertical boundary _ = 2 overlap error

Minimum Error Boundary Cut

The Image Quilting Algorithm –Pick size of block and size of overlap –Synthesize blocks in raster order –Search input texture for block that satisfies overlap constraints (above and left) Easy to optimize using NN search [Liang et.al., ’ 01] –Paste new block into resulting texture Compute minimal error boundary cut

Synthesis Results

input image Portilla & Simoncelli Wei & LevoyImage Quilting Xu, Guo & Shum

Portilla & Simoncelli Wei & LevoyImage Quilting Xu, Guo & Shum input image

Portilla & Simoncelli Wei & LevoyImage Quilting input image Xu, Guo & Shum

Failures

Image Quilting vs. Graph Cut Input Image Quilting Graph Cut (siggraph 03’)

Texture Transfer + Luminance Constraint

+= += parmesan rice

Conclusion –No multi-scale, no one-pixel-at-a-time! –fast and very simple –Improved stability –Results are not bad

Pixel-based Methods Compare local causal neighbourhoods Efros and Leung (ICCV ’ 99) Wei and Levoy (Siggraph 2000) Ashikhmin (I3D 2001) InputOutput

Patch-based Methods Copy patches of pixels rather than single pixels Chaos Mosaic, Xu et al, 1997 Patch-Based Sampling, Liang et al(ACM 2001) Image Quilting, Efros and Williams(Siggraph 2001)