Image Mix and Match Internet images – Colorization – Most similar image searching – Collage Object insertion

Data-driven approach for robust similarity measure – Cross domain(Photo, Photo with different lighting, Painting) – No domain specific treatments 異種画像に利用できる画像の類似度計算法

Idea – Detect unique region of the target image (comparing to the other) – Place high weight on the unique regions – 与えられた画像のどの特徴が， Web 上の膨大な量の画像に対してユ ニークかを学習 – ユニークな画像特徴に重みを置く

Image feature vector ( 画像特徴ベクトル ) – Intensity histogram, Gradient Magnitude Histogram, HoG, SIFT – 画像間距離は，この特徴ベクトル間の距離として計算する事が多い Intensity histogram HoG, Histogram of oriented gradient ， 4k-5k dimension Linear support vector machine (see Pattern recognition textbook) Given d-dimensional feature vectors belongs to class A and B, xi ∊ A, yi ∊ B xi, yi ∊ R d Find the maximum-margin hyperplane that divides xi ∊ A, yi ∊ B この絵は２Ｄだけど本当は特徴ベクトルと同じだ けの次元， 5000 次元とか xixi yiyi

Goal Given a image I p – Detect unique parts of feature vector of I p comparing to the others – Place high weight on the unique regions IpIp Other images on Web 1) Compute feature vectors of I p x p and the other internet images x i 2) Compute hiperplane by linear SVM 3) Project feature vectors of all images onto the normal of the hiperplane この normal が PCA の軸のような役割になる

Image colorization from internet image Gray scale 画像の色付けを， Web 上の画像を参照して行う – Combination of many techniques Internet image search, foreground segmentation, suitable image filtering, Image similarity measure, graph-based color transfer, selection UI for weight tuning Input: Image + text label (e.g. rooster)

Procedure – Input: grayscale image with foreground segmentation & text label – Search images from internet Google image search / Flickr Automatic fore ground extraction Filter similar images for back/fore ground(Ad hoc energy function intensity, texture, density of SIFT) – Color transfer Graph based color transfer method – Maintain the neighborhood consistency Compute with different weighting values  The user can select one of them – Output: Colored images with different reference images and different weighting values

Color transfter の計算時に， gray 画像と参照画像の両方とも，微小領域に分割し，近傍に矛盾がないような color transfer を計算している．例えば蝶の羽などでは，黄色の隣にはオレンジ色が来る事は無いなど，良い結果を生んでいる．

Arcimboldo-style collage generation – Input : Source image and text label for searching element image cutouts – Output: collage consists from internet images

Giuseppe Arcimboldo Itary Collage like drawing – Each element is recognizable (elements are taken from a same theme) – The assembly of the elements resembles something

Two problems Best fitting cutout search Input image segmentation 1) Mean shift clustering Compute modes in color & space feature space エッジ保存フィルタを連続して書けるような物 2) Marge & split strange local regions -Mean shift clustering generates local regions that not match any element cutout image from internet -Trim such regions by Ad hoc iteration 1) Search images from internet 2) Cutout foreground image by saliency detection and GrabCut 3) Distance metric between - Hole in the target image - Cutout image from internet Color distance term / Shape distance term With best fitting affine transformation Semantic aware segmentation is difficult…

New tool for inserting objects into Photographs – geometry and light estimation with user’s guide Less user interaction – Deal with interior light and exterior lights (e.g. sun light from window) – ある点がすごい新しいとかではなく，他と比較して全体的なパフォーマ ンスが上がっている感じ Inputs – Single image – User annotation (geometry, light source position) – 3D model that will be inserted into the Image

Overview of the system Geometry estimation – Previous work + user’s correction, user interface to add other geometry Light source estimation – Next page Object insertion – Add object into 3D scene and render it with estimated parameter

Interior light estimation 1) Decompose input image into Albedo and direct light image 2) User points the position of the light 3) Automatically adjust light parameter (position & RGB) !!Full automatic light source detection from single image is very difficult!! =+ Exterior light estimation 1)The user marks boundary of the source and projection of the exterior light 2) The system automatically computes mask and direction

今年の SIGGRAPH に Image = Albedo + Direct Image = Albedo + Indirect + Direct という分解をする論文あった

Results