Download presentation
Presentation is loading. Please wait.
1
Computer Vision Segmentation Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, T. Darrel,...
2
Computer Vision 2 Should have been last week ;-) Sequential structure from motion Initialize motion from two images Initialize structure For each additional view –Determine pose of camera –Refine and extend structure Refine structure and motion
3
Computer Vision 3 Initial projective camera motion Choose P and P´compatible with F Reconstruction up to projective ambiguity (reference plane;arbitrary) Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion Same for more views? different projective basis
4
Computer Vision 4 Initializing projective structure Reconstruct matches in projective frame by minimizing the reprojection error Non-iterative optimal solution Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion
5
Computer Vision 5 Projective pose estimation Infere 2D-3D matches from 2D-2D matches Compute pose from (RANSAC,6pts) F X x Inliers: Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion
6
Computer Vision 6 Refining structure Extending structure 2-view triangulation (Iterative linear) Initialize motion Initialize structure For each additional view Determine pose of camera Refine and extend structure Refine structure and motion Refining and extending structure
7
Computer Vision 7 Refining structure and motion use bundle adjustment Also model radial distortion to avoid bias!
8
Computer Vision 8
9
Computer Vision 9 Application: video augmentation
10
Computer Vision 10 Jan 16/18-Introduction Jan 23/25CamerasRadiometry Jan 30/Feb1Sources & ShadowsColor Feb 6/8Linear filters & edgesTexture Feb 13/15Multi-View GeometryStereo Feb 20/22Optical flowProject proposals Feb27/Mar1Affine SfMProjective SfM Mar 6/8Camera CalibrationSegmentation Mar 13/15Springbreak Mar 20/22FittingProb. Segmentation Mar 27/29Silhouettes and Photoconsistency Linear tracking Apr 3/5Project UpdateNon-linear Tracking Apr 10/12Object Recognition Apr 17/19Range data Apr 24/26Final project Tentative class schedule
11
Computer Vision 11 Segmentation and Grouping Motivation: not information is evidence Obtain a compact representation from an image/motion sequence/set of tokens Should support application Broad theory is absent at present Grouping (or clustering) –collect together tokens that “belong together” Fitting –associate a model with tokens –issues which model? which token goes to which element? how many elements in the model?
12
Computer Vision 12 General ideas tokens –whatever we need to group (pixels, points, surface elements, etc., etc.) top down segmentation –tokens belong together because they lie on the same object bottom up segmentation –tokens belong together because they are locally coherent These two are not mutually exclusive
13
Computer Vision 13 Why do these tokens belong together?
14
Computer Vision 14
15
Computer Vision 15 Basic ideas of grouping in humans Figure-ground discrimination –grouping can be seen in terms of allocating some elements to a figure, some to ground –impoverished theory Gestalt properties –elements in a collection of elements can have properties that result from relationships (Muller-Lyer effect) gestaltqualitat –A series of factors affect whether elements should be grouped together Gestalt factors
16
Computer Vision 16
17
Computer Vision 17
18
Computer Vision 18
19
Computer Vision 19
20
Computer Vision 20
21
Computer Vision 21
22
Computer Vision 22
23
Computer Vision 23 Technique: Shot Boundary Detection Find the shots in a sequence of video –shot boundaries usually result in big differences between succeeding frames Strategy: –compute interframe distances –declare a boundary where these are big Possible distances –frame differences –histogram differences –block comparisons –edge differences Applications: –representation for movies, or video sequences find shot boundaries obtain “most representative” frame –supports search
24
Computer Vision 24 Technique: Background Subtraction If we know what the background looks like, it is easy to identify “interesting bits” Applications –Person in an office –Tracking cars on a road –surveillance Approach: –use a moving average to estimate background image –subtract from current frame –large absolute values are interesting pixels trick: use morphological operations to clean up pixels
25
Computer Vision 25
26
Computer Vision 26
27
Computer Vision 27
28
Computer Vision 28
29
Computer Vision 29
30
Computer Vision 30
31
Computer Vision 31
32
Computer Vision 32
33
Computer Vision 33
34
Computer Vision 34 Segmentation as clustering Cluster together (pixels, tokens, etc.) that belong together Agglomerative clustering –attach closest to cluster it is closest to –repeat Divisive clustering –split cluster along best boundary –repeat Point-Cluster distance –single-link clustering –complete-link clustering –group-average clustering Dendrograms –yield a picture of output as clustering process continues
35
Computer Vision 35 Simple clustering algorithms
36
Computer Vision 36
37
Computer Vision 37 K-Means Choose a fixed number of clusters Choose cluster centers and point-cluster allocations to minimize error can’t do this by search, because there are too many possible allocations.
38
Computer Vision 38 K-means clustering using intensity alone and color alone Image Clusters on intensity Clusters on color
39
Computer Vision 39 K-means using color alone, 11 segments Image Clusters on color
40
Computer Vision 40 K-means using color alone, 11 segments.
41
Computer Vision 41 K-means using colour and position, 20 segments
42
Computer Vision 42 Graph theoretic clustering Represent tokens using a weighted graph. –affinity matrix Cut up this graph to get subgraphs with strong interior links
43
Computer Vision 43
44
Computer Vision 44
45
Computer Vision 45
46
Computer Vision 46
47
Computer Vision 47
48
Computer Vision 48 Measuring Affinity Intensity Texture Distance
49
Computer Vision 49 Scale affects affinity
50
Computer Vision 50 Eigenvectors and cuts Simplest idea: we want a vector a giving the association between each element and a cluster We want elements within this cluster to, on the whole, have strong affinity with one another We could maximize But need the constraint This is an eigenvalue problem - choose the eigenvector of A with largest eigenvalue
51
Computer Vision 51 Example eigenvector points matrix eigenvector
52
Computer Vision 52
53
Computer Vision 53 More than two segments Two options –Recursively split each side to get a tree, continuing till the eigenvalues are too small –Use the other eigenvectors
54
Computer Vision 54 More than two segments
55
Computer Vision 55 Normalized cuts Current criterion evaluates within cluster similarity, but not across cluster difference Instead, we’d like to maximize the within cluster similarity compared to the across cluster difference Write graph as V, one cluster as A and the other as B Maximize i.e. construct A, B such that their within cluster similarity is high compared to their association with the rest of the graph
56
Computer Vision 56
57
Computer Vision 57
58
Computer Vision 58 Normalized cuts Write a vector y whose elements are 1 if item is in A, -b if it’s in B Write the matrix of the graph as W, and the matrix which has the row sums of W on its diagonal as D, 1 is the vector with all ones. Criterion becomes and we have a constraint This is hard to do, because y’s values are quantized
59
Computer Vision 59
60
Computer Vision 60 Normalized cuts Instead, solve the generalized eigenvalue problem which gives Now look for a quantization threshold that maximises the criterion --- i.e all components of y above that threshold go to one, all below go to -b
61
Computer Vision 61 Figure from “Image and video segmentation: the normalised cut framework”, by Shi and Malik, copyright IEEE, 1998
62
Computer Vision 62 Figure from “Normalized cuts and image segmentation,” Shi and Malik, copyright IEEE, 2000
63
Computer Vision 63 Image parsing: Unifying Segmentation, Detection and Recognition Tu, Chen, Yuille & Zhu ICCV’03
64
Computer Vision 64 Image parsing Generative models for each class some random samples for 5 and H some random face samples (PCA model)
65
Computer Vision 65 DDMCMC Data Driven Markov Chain Monte Carlo
66
Computer Vision 66 Image parsing
67
Computer Vision 67 Image parsing
68
Computer Vision 68 Next class: Fitting Reading: Chapter 15
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.