Chapter 14: SEGMENTATION BY CLUSTERING 1. 2 Outline Introduction Human Vision & Gestalt Properties Applications – Background Subtraction – Shot Boundary.

Slides:



Advertisements
Similar presentations
Partitional Algorithms to Detect Complex Clusters
Advertisements

Segmentácia farebného obrazu
Normalized Cuts and Image Segmentation
Online Social Networks and Media. Graph partitioning The general problem – Input: a graph G=(V,E) edge (u,v) denotes similarity between u and v weighted.
I Images as graphs Fully-connected graph – node for every pixel – link between every pair of pixels, p,q – similarity w ij for each link j w ij c Source:
Segmentation Course web page: vision.cis.udel.edu/~cv May 2, 2003  Lecture 29.
Human-Computer Interaction Human-Computer Interaction Segmentation Hanyang University Jong-Il Park.
10/11/2001Random walks and spectral segmentation1 CSE 291 Fall 2001 Marina Meila and Jianbo Shi: Learning Segmentation by Random Walks/A Random Walks View.
Lecture 6 Image Segmentation
Image segmentation. The goals of segmentation Group together similar-looking pixels for efficiency of further processing “Bottom-up” process Unsupervised.
Lecture 21: Spectral Clustering
Segmentation. Terminology Segmentation, grouping, perceptual organization: gathering features that belong together Fitting: associating a model with observed.
Normalized Cuts and Image Segmentation Jianbo Shi and Jitendra Malik, Presented by: Alireza Tavakkoli.
Region Segmentation. Find sets of pixels, such that All pixels in region i satisfy some constraint of similarity.
Image Segmentation Chapter 14, David A. Forsyth and Jean Ponce, “Computer Vision: A Modern Approach”.
Segmentation and Clustering. Segmentation: Divide image into regions of similar contentsSegmentation: Divide image into regions of similar contents Clustering:
Segmentation CSE P 576 Larry Zitnick Many slides courtesy of Steve Seitz.
Segmentation Divide the image into segments. Each segment:
Segmentation Kyongil Yoon. Segmentation Obtain a compact representation of what is helpful (in the image) No comprehensive theory of segmentation Human.
Normalized Cuts Demo Original Implementation from: Jianbo Shi Jitendra Malik Presented by: Joseph Djugash.
Segmentation Graph-Theoretic Clustering.
Computer Vision Segmentation Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, T. Darrel,...
Segmentation by Clustering Reading: Chapter 14 (skip 14.5) Data reduction - obtain a compact representation for interesting image data in terms of a set.
Image Segmentation Some slides: courtesy of O. Capms, Penn State, J.Ponce and D. Fortsyth, Computer Vision Book.
Image Segmentation A Graph Theoretic Approach. Factors for Visual Grouping Similarity (gray level difference) Similarity (gray level difference) Proximity.
Perceptual Organization: Segmentation and Optical Flow.
Computer Vision Segmentation Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, T. Darrel,...
Segmentation and Perceptual Grouping The problem Gestalt Edge extraction: grouping and completion Image segmentation.
Computer Vision - A Modern Approach Set: Segmentation Slides by D.A. Forsyth Segmentation and Grouping Motivation: not information is evidence Obtain a.
Clustering Unsupervised learning Generating “classes”
Graph-based Segmentation
Graph-based consensus clustering for class discovery from gene expression data Zhiwen Yum, Hau-San Wong and Hongqiang Wang Bioinformatics, 2007.
Image Segmentation Rob Atlas Nick Bridle Evan Radkoff.
MASKS © 2004 Invitation to 3D vision Lecture 3 Image Primitives andCorrespondence.
Segmentation Techniques Luis E. Tirado PhD qualifying exam presentation Northeastern University.
Presenter : Kuang-Jui Hsu Date : 2011/5/3(Tues.).
Segmentation using eigenvectors
CSSE463: Image Recognition Day 34 This week This week Today: Today: Graph-theoretic approach to segmentation Graph-theoretic approach to segmentation Tuesday:
Segmentation by Clustering
Segmentation using eigenvectors Papers: “Normalized Cuts and Image Segmentation”. Jianbo Shi and Jitendra Malik, IEEE, 2000 “Segmentation using eigenvectors:
Computer Vision Lecture 5. Clustering: Why and How.
Recognition using Regions (Demo) Sudheendra V. Outline Generating multiple segmentations –Normalized cuts [Ren & Malik (2003)] Uniform regions –Watershed.
Edge Linking & Boundary Detection
Segmentation Course web page: vision.cis.udel.edu/~cv May 7, 2003  Lecture 31.
CSE 185 Introduction to Computer Vision Pattern Recognition 2.
CS654: Digital Image Analysis Lecture 25: Hough Transform Slide credits: Guillermo Sapiro, Mubarak Shah, Derek Hoiem.
Segmentation & Grouping Tuesday, Sept 23 Kristen Grauman UT-Austin.
Image Segmentation Shengnan Wang
Image Segmentation Superpixel methods Speaker: Hsuan-Yi Ko.
 In the previews parts we have seen some kind of segmentation method.  In this lecture we will see graph cut, which is a another segmentation method.
Mestrado em Ciência de Computadores Mestrado Integrado em Engenharia de Redes e Sistemas Informáticos VC 15/16 – TP10 Advanced Segmentation Miguel Tavares.
Representing Moving Images with Layers J. Y. Wang and E. H. Adelson MIT Media Lab.
Spectral Clustering Shannon Quinn (with thanks to William Cohen of Carnegie Mellon University, and J. Leskovec, A. Rajaraman, and J. Ullman of Stanford.
1 Review and Summary We have covered a LOT of material, spending more time and more detail on 2D image segmentation and analysis, but hopefully giving.
MASKS © 2004 Invitation to 3D vision Lecture 3 Image Primitives andCorrespondence.
Normalized Cuts and Image Segmentation Patrick Denis COSC 6121 York University Jianbo Shi and Jitendra Malik.
Grouping and Segmentation Previously: model-based grouping (e.g., straight line, figure 8…) Now: general ``bottom-up” image organization (Like detecting-specific-brightness-patterns.
Document Clustering with Prior Knowledge Xiang Ji et al. Document Clustering with Prior Knowledge. SIGIR 2006 Presenter: Suhan Yu.
K-Means Segmentation.
Miguel Tavares Coimbra
Segmentation by clustering: normalized cut
Exercise class 13 : Image Segmentation
CSSE463: Image Recognition Day 34
Segmentation and Grouping
Segmentation Graph-Theoretic Clustering.
Perceiving and Recognizing Objects
Grouping.
Emel Doğrusöz Esra Ataer Muhammet Baştan Tolga Can
Spectral Clustering Eric Xing Lecture 8, August 13, 2010
CSSE463: Image Recognition Day 34
Presentation transcript:

Chapter 14: SEGMENTATION BY CLUSTERING 1

2

Outline Introduction Human Vision & Gestalt Properties Applications – Background Subtraction – Shot Boundary Detection Segmentation by Clustering – Simple Clustering Methods – K-means Segmentation by Graph-Theoretic Clustering – The Overall Approach – Affinity Measures – Normalized Cut 3

Introduction 4

General Ideas Tokens – Whatever we need to group (pixels, points, surface elements, etc., etc.) Top down segmentation – Tokens belong together because they lie on the same object Bottom up segmentation – Tokens belong together because they are locally coherent 5

Why & What is segmentation – Emphasize the property – Make main idea interesting – There are too many pixels in a single image – We need some form of compact, summary representation 6

What segmentation can do – Summarizing video – Finding machined parts – Finding people – Finding buildings in satellite images – Searching a collection of images 7

Model Problems – Forming image segments Roughly coherent color and texture – Finding body segments Arms, torso, etc. – Fitting lines to edge points To fit a line to a set of points 8

Segmentation as Clustering Partitioning – Decompose an image into regions – Decompose an image into extended blobs – Take a video sequence and decompose it into shots Grouping 9

Human Vision & Gestalt Properties 10

Figure-ground discrimination – Grouping can be seen in terms of allocating some elements to a figure, some to ground – Can be based on local bottom-up cues or high level recognition 11

Figure-ground discrimination 12

A series of factors affect whether elements should be grouped together 13

A series of factors affect whether elements should be grouped together 14

15

16

Grouping by occlusions 17

Grouping by invisible completions 18

Outline Introduction Human Vision & Gestalt Properties Applications – Background Subtraction – Shot Boundary Detection Segmentation by Clustering – Simple Clustering Methods – K-means Segmentation by Graph-Theoretic Clustering – The Overall Approach – Affinity Measures – Normalized Cut 19

Application Background Subtraction Anything that doesn’t look like a known background is interesting 20

Take a picture? Not good… The background changes slowly overtime… 21

Estimate the value of background pixels using a moving average. Ideally, the moving average should track the changes in the background. 22

23

Shot Boundary Detection Where substantial changes in a video are interesting. 24

25

Computing distance – Frame differencing – Histogram-based – Block comparison – Edge differencing pixel-by-pixel histogram-by-histogram box-by-box edge map-by-edge map 26

Outline Introduction Human Vision & Gestalt Properties Applications – Background Subtraction – Shot Boundary Detection Segmentation by Clustering – Simple Clustering Methods – K-means Segmentation by Graph-Theoretic Clustering – The Overall Approach – Affinity Measures – Normalized Cut 27

Segmentation by Clustering Simple Clustering Methods Divisive clustering 28

Divisive clustering 29 p, q, r, s, t r, s, t p, qqp r s, t st

Agglomerative clustering 30

Agglomerative clustering 31 p, q, r, s, t qp p, q r r, s, t st s, t

Q: what the distance of A: 32

Single-link clustering 33

Complete-link clustering 34

Group average clustering 35

K-means 36

37

38

Outline Introduction Human Vision & Gestalt Properties Applications – Background Subtraction – Shot Boundary Detection Segmentation by Clustering – Simple Clustering Methods – K-means Segmentation by Graph-Theoretic Clustering – The Overall Approach – Affinity Measures – Normalized Cut 39

Segmentation by Graph-theoretic Clustering Terminology for Graph – Vertices – Edges Directed graph Undirected graph Weighted graph Self-loop Connected graph – Connected components

Weighted graph represented by a square matrix The Overall Approach 41 Undirected weighted graph Weighted matrix

42 Set of well separated pointsAffinity matrix

Affinity Measures Affinity by Distance 43

Affinity by Intensity Affinity by Color Affinity by Texture 44

Eigenvectors and segmentation 45 Extracting a Single Good Cluster { association of element I with cluster n } × { affinity between I and j } × { association of element j with cluster n} :Element affinities :The vector of weight linking elements to the th cluster

46 Scaling by scales Normalizes the weights by requiring that Differentiation and dropping :eigenvector of Lagrangian

47 Small

Extracting Weights for a Set of Clusters – quite tight – distinct. 48 The three largest eigenvalues of the affinity matrix for the dataset

49

Outline Introduction Human Vision & Gestalt Properties Applications – Background Subtraction – Shot Boundary Detection Segmentation by Clustering – Simple Clustering Methods – K-means Segmentation by Graph-Theoretic Clustering – The Overall Approach – Affinity Measures – Normalized Cut 50

Normalized Cut 51

Association of set: A B 52 Normalized cut: Cut of set: A B A B Minimize

Minimize Ncut – Transform Ncut equation to a matrical form NP-Hard!

Real Vector & Threshold If y is allowed to take real values then the minimization can be done by solving the generalized eigenvalue system Note that,so the first eigenvector is y=I with eigenvalue 0. The second smallest eigenvector is the real valued solution to this problem! Choosing a threshold to decide that nodes are assigned to one side or the other.

Summarize 1.Define a similarity function between 2 nodes. 2.Compute affinity matrix ( A ) and degree matrix (D). 3.Solve 4.Use the eigenvector with the second smallest eigenvalue to bipartition the graph. 5.Recursively partition the segmented parts if necessary. 55

Example 56

57

Thank you 58