Fast search methods Pasi Fränti Clustering methods: Part 5 Speech and Image Processing Unit School of Computing University of Eastern Finland 5.5.2014.

Slides:

Advertisements

Similar presentations

K-MEANS Michael Jones ENEE698Q Fall Overview  Introduction  Problem Formulation  How K-Means Works  Pros and Cons of Using K-Means  How to.

Advertisements

Variable Metric For Binary Vector Quantization UNIVERSITY OF JOENSUU DEPARTMENT OF COMPUTER SCIENCE JOENSUU, FINLAND Ismo Kärkkäinen and Pasi Fränti.

CMU SCS : Multimedia Databases and Data Mining Lecture #7: Spatial Access Methods - Metric trees C. Faloutsos.

電腦視覺 Computer and Robot Vision I

Searching on Multi-Dimensional Data

Fast Algorithm for Nearest Neighbor Search Based on a Lower Bound Tree Yong-Sheng Chen Yi-Ping Hung Chiou-Shann Fuh 8 th International Conference on Computer.

Ilias Theodorakopoulos PhD Candidate

Special Topic on Image Retrieval Local Feature Matching Verification.

Registration of two scanned range images using k-d tree accelerated ICP algorithm By Xiaodong Yan Dec

Fast High-Dimensional Feature Matching for Object Recognition David Lowe Computer Science Department University of British Columbia.

Computational Support for RRTs David Johnson. Basic Extend.

CSE 589 Applied Algorithms Spring 1999 Image Compression Vector Quantization Nearest Neighbor Search.

Object retrieval with large vocabularies and fast spatial matching

Clustering… in General In vector space, clusters are vectors found within  of a cluster vector, with different techniques for determining the cluster.

Lecture 28: Bag-of-words models

Scalable Data Mining The Auton Lab, Carnegie Mellon University Brigham Anderson, Andrew Moore, Dan Pelleg, Alex Gray, Bob Nichols, Andy.

Bag-of-features models

Adapted by Doug Downey from Machine Learning EECS 349, Bryan Pardo Machine Learning Clustering.

Nearest Neighbour Condensing and Editing David Claus February 27, 2004 Computer Vision Reading Group Oxford.

Part 3 Vector Quantization and Mixture Density Model CSE717, SPRING 2008 CUBS, Univ at Buffalo.

FLANN Fast Library for Approximate Nearest Neighbors

Ch 5 Practical Point Pattern Analysis Spatial Stats & Data Analysis by Magdaléna Dohnalová.

Fast vector quantization image coding by mean value predictive algorithm Authors: Yung-Gi Wu, Kuo-Lun Fan Source: Journal of Electronic Imaging 13(2),

Review: Intro to recognition Recognition tasks Machine learning approach: training, testing, generalization Example classifiers Nearest neighbor Linear.

Silvio Cesare Ph.D. Candidate, Deakin University.

Image Based Positioning System Ankit Gupta Rahul Garg Ryan Kaminsky.

What Is the Most Efficient Way to Select Nearest Neighbor Candidates for Fast Approximate Nearest Neighbor Search? Masakazu Iwamura, Tomokazu Sato and.

Clustering Methods: Part 2d Pasi Fränti Speech & Image Processing Unit School of Computing University of Eastern Finland Joensuu, FINLAND Swap-based.

Clustering methods Course code: Pasi Fränti Speech & Image Processing Unit School of Computing University of Eastern Finland Joensuu,

Self-organizing map Speech and Image Processing Unit Department of Computer Science University of Joensuu, FINLAND Pasi Fränti Clustering Methods: Part.

Cut-based & divisive clustering Clustering algorithms: Part 2b Pasi Fränti Speech & Image Processing Unit School of Computing University of Eastern.

Non-Parameter Estimation 主講人：虞台文. Contents Introduction Parzen Windows k n -Nearest-Neighbor Estimation Classification Techiques – The Nearest-Neighbor.

Genetic Algorithm Using Iterative Shrinking for Solving Clustering Problems UNIVERSITY OF JOENSUU DEPARTMENT OF COMPUTER SCIENCE FINLAND Pasi Fränti and.

Machine Learning Neural Networks (3). Understanding Supervised and Unsupervised Learning.

FAST DYNAMIC QUANTIZATION ALGORITHM FOR VECTOR MAP COMPRESSION Minjie Chen, Mantao Xu and Pasi Fränti University of Eastern Finland.

Data Extraction using Image Similarity CIS 601 Image Processing Ajay Kumar Yadav.

1 Pattern Recognition Pattern recognition is: 1. A research area in which patterns in data are found, recognized, discovered, …whatever. 2. A catchall.

David Claus and Christoph F. Eick: Nearest Neighbor Editing and Condensing Techniques Nearest Neighbor Editing and Condensing Techniques 1.Nearest Neighbor.

1 CSE 552/652 Hidden Markov Models for Speech Recognition Spring, 2006 Oregon Health & Science University OGI School of Science & Engineering John-Paul.

Chapter 9 DTW and VQ Algorithm  9.1 Basic idea of DTW  9.2 DTW algorithm  9.3 Basic idea of VQ  9.4 LBG algorithm  9.5 Improvement of VQ.

Smooth Side-Match Classified Vector Quantizer with Variable Block Size IEEE Transaction on image processing, VOL. 10, NO. 5, MAY 2001 Department of Applied.

Planar-Oriented Ripple Based Greedy Search Algorithm for Vector Quantization Presenter: Tzu-Meng Huang Adviser:Dr. Yeou-Jiunn Chen Date:2011/11/ /11/161.

Genetic algorithms (GA) for clustering Pasi Fränti Clustering Methods: Part 2e Speech and Image Processing Unit School of Computing University of Eastern.

Outline Transmitters (Chapters 3 and 4, Source Coding and Modulation) (week 1 and 2) Receivers (Chapter 5) (week 3 and 4) Received Signal Synchronization.

Optimal Component Analysis Optimal Linear Representations of Images for Object Recognition X. Liu, A. Srivastava, and Kyle Gallivan, “Optimal linear representations.

Spatial Subdivision Techniques SAMPL Group Presentation By Gerald Dalley.

A Fast LBG Codebook Training Algorithm for Vector Quantization Presented by 蔡進義.

Chapter 4: Feature representation and compression

K-Means clustering accelerated algorithms using the triangle inequality Ottawa-Carleton Institute for Computer Science Alejandra Ornelas Barajas School.

Fast Query-Optimized Kernel Machine Classification Via Incremental Approximate Nearest Support Vectors by Dennis DeCoste and Dominic Mazzoni International.

A new clustering tool of Data Mining RAPID MINER.

Clustering Algorithms Sunida Ratanothayanon. What is Clustering?

Example Apply hierarchical clustering with d min to below data where c=3. Nearest neighbor clustering d min d max will form elongated clusters!

Iterative K-Means Algorithm Based on Fisher Discriminant UNIVERSITY OF JOENSUU DEPARTMENT OF COMPUTER SCIENCE JOENSUU, FINLAND Mantao Xu to be presented.

BLOCK BASED MOTION ESTIMATION. Road Map Block Based Motion Estimation Algorithms. Procedure Of 3-Step Search Algorithm. 4-Step Search Algorithm. N-Step.

Debrup Chakraborty Non Parametric Methods Pattern Recognition and Machine Learning.

Registration and Alignment Speaker: Liuyu

How to cluster data Algorithm review Extra material for DAA Prof. Pasi Fränti Speech & Image Processing Unit School of Computing University.

Finding Clusters within a Class to Improve Classification Accuracy Literature Survey Yong Jae Lee 3/6/08.

Genetic Algorithms for clustering problem Pasi Fränti

Machine Learning Lecture 4: Unsupervised Learning (clustering) 1.

Agglomerative clustering (AC)

Advisor: Chang, Chin-Chen Student: Chen, Chang-Chu

Fast nearest neighbor searches in high dimensions Sami Sieranoja

Random Swap algorithm Pasi Fränti

Machine Learning University of Eastern Finland

Random Swap algorithm Pasi Fränti

Pasi Fränti and Sami Sieranoja

Course project work tasks

Clustering methods: Part 10

Presentation transcript:

Fast search methods Pasi Fränti Clustering methods: Part 5 Speech and Image Processing Unit School of Computing University of Eastern Finland

Methods considered Classical speed-up techniques Partial distortion search (PDS) Mean distance ordered partial search (MPS) Speed-up of k-means Reduced-search based on centroid activity External search data structures Nearest neighbor graph Kd-tree

Partial distortion search (PDS) [Bei and Gray, 1985: IEEE Trans. Communications] Current best candidate gives upper limit. Distances calculated cumulatively. After each addition, check if the partial distortion exceeds the smallest distance found so far. If it exceeds, then terminate the search.

Calculate distance along projection axis. If distance is outside bounding circle defined by the best candidate, drop the vector. Mean-distance ordered partial search (MPS) [Ra and Kim, 1993: IEEE Trans. Circuits and Systems]

Bounds of the MPS method Input vector Best candidate Bound

Pseudo code of MPS search This should be updated according to what was said during lectures!!

Activity classification [Kaukoranta et al., 2000: IEEE Trans. Image Processing]

Reduced search based on activity classification

Classification due to iterations

Activity of vectors in Random Swap

Effect on distance calculations K-means K-means with activity classification

Effect on processing time For improving K-means algorithm 3.8 % 1.6 %

Comparison of speed-up methods

Improvement of reduced search

Neighborhood graph Full search: O(N) distance calculations. Graph structure: O(k) distance calculations. Full search:Graph structure:

Sample graph structure

K-d tree See the course: Design of Spatial Information Systems

Literature 1.T. Kaukoranta, P. Fränti and O. Nevalainen, "A fast exact GLA based on code vector activity detection", IEEE Trans. on Image Processing, 9 (8), , August C.-D. Bei and R.M. Gray, "An improvement of the minimum distortion encoding algorithm for vector quantization", IEEE Transactions on Communications, 33 (10), , October S.-W. Ra and J.-K. Kim, "A Fast Mean-Distance-Ordered Partial Codebook Search Algorithm for Image Vector Quantization", IEEE Transactions on Circuits and Systems, 40 (9), , Sebtember J.Z.C. Lai, Y.-C.Liaw, J.Liu, "Fast k-nearest-neighbor search based on projection and triangular inequality", Pattern Recognition, 40, , C. Elkan. Using the Triangle Inequality to Accelerate k-Means. Int. Conf. on Machine Learning, (ICML'03), pp

6.James McNames, "A fast nearest neighbor algorithm based on a principal axis search tree", IEEE Trans. on Pattern Analysis and Machine Intelligence, 23(9): , September J.H. Friedman, J.L. Bentley and R.A. Finkel, "An algorithm for finding best matches in logarithmic expected time," ACM Trans. on Mathematical Software, 3 (3), pp , September R. Sproull, "Refinements to nearest-neighbor searching in K-d tree," Algorithmica, 6, pp , Literature