Clustering Art & Learning the Semantics of Words and Pictures Manigantan Sethuraman.

Slides:

Advertisements

Similar presentations

LEARNING SEMANTICS OF WORDS AND PICTURES TEJASWI DEVARAPALLI.

Advertisements

Algorithms of Google News An Analysis of Google News Personalization Scalable Online Collaborative Filtering 1.

Big Ideas in Cmput366. Search Blind Search State space representation Iterative deepening Heuristic Search A*, f(n)=g(n)+h(n), admissible heuristics Local.

Hierarchical Clustering, DBSCAN The EM Algorithm

PARTITIONAL CLUSTERING

Image Retrieval Basics Uichin Lee KAIST KSE Slides based on “Relevance Models for Automatic Image and Video Annotation & Retrieval” by R. Manmatha (UMASS)

How the edges of a line, paragraph, object, or table are positioned horizontally and vertically between the margins or on a page.

Unsupervised learning

Real-Time Human Pose Recognition in Parts from Single Depth Images Presented by: Mohammad A. Gowayyed.

Chapter 11 Beyond Bag of Words. Question Answering n Providing answers instead of ranked lists of documents n Older QA systems generated answers n Current.

Li-Jia Li Yongwhan Lim Li Fei-Fei Chong Wang David M. Blei B UILDING AND U SING A S EMANTIVISUAL I MAGE H IERARCHY CVPR, 2010.

Database Management Systems, R. Ramakrishnan1 Computing Relevance, Similarity: The Vector Space Model Chapter 27, Part B Based on Larson and Hearst’s slides.

Expectation Maximization for GMM Comp344 Tutorial Kai Zhang.

Ranking by Odds Ratio A Probability Model Approach let be a Boolean random variable: document d is relevant to query q otherwise Consider document d as.

Chapter 1: Data Collection

Region Based Image Annotation Through Multiple-Instance Learning By: Changbo Yang Wayne State University Department of Computer Science.

BLOSUM Information Resources Algorithms in Computational Biology Spring 2006 Created by Itai Sharon.

Multiple Object Class Detection with a Generative Model K. Mikolajczyk, B. Leibe and B. Schiele Carolina Galleguillos.

Clustering Unsupervised learning Generating “classes”

Information Retrieval in Practice

Modeling and Finding Abnormal Nodes (chapter 2) 駱宏毅 Hung-Yi Lo Social Network Mining Lab Seminar July 18, 2007.

Entropy and some applications in image processing Neucimar J. Leite Institute of Computing

ERC StG: Multilingual Joint Word Sense Disambiguation (MultiJEDI) Roberto Navigli 1 A Graph-based Algorithm for Inducing Lexical Taxonomies from Scratch.

Disambiguation of References to Individuals Levon Lloyd (State University of New York) Varun Bhagwan, Daniel Gruhl (IBM Research Center) Varun Bhagwan,

CHAMELEON : A Hierarchical Clustering Algorithm Using Dynamic Modeling

Edit by Dalton Lin Copyright © 2012 Elsevier Inc. All rights reserved.. Chapter 1 Vision, the Challenge.

Retrieval Effectiveness of an Ontology-based Model for Information Selection Khan, L., McLeod, D. & Hovy, E. Presented by Danielle Lee.

Image Recognition using Hierarchical Temporal Memory Radoslav Škoviera Ústav merania SAV Fakulta matematiky, fyziky a informatiky UK.

Exploiting Ontologies for Automatic Image Annotation M. Srikanth, J. Varner, M. Bowden, D. Moldovan Language Computer Corporation

Research Interests Georgia Koloniari Computer Science Department University of Ioannina, Greece.

PageRank for Product Image Search Kevin Jing (Googlc IncGVU, College of Computing, Georgia Institute of Technology) Shumeet Baluja (Google Inc.) WWW 2008.

Introduction to Data Mining Group Members: Karim C. El-Khazen Pascal Suria Lin Gui Philsou Lee Xiaoting Niu.

UOS 1 Ontology Based Personalized Search Zhang Tao The University of Seoul.

10/22/2015ACM WIDM'20051 Semantic Similarity Methods in WordNet and Their Application to Information Retrieval on the Web Giannis Varelas Epimenidis Voutsakis.

Video Google: A Text Retrieval Approach to Object Matching in Videos Josef Sivic and Andrew Zisserman.

1 Computing Relevance, Similarity: The Vector Space Model.

Web Image Retrieval Re-Ranking with Relevance Model Wei-Hao Lin, Rong Jin, Alexander Hauptmann Language Technologies Institute School of Computer Science.

Unsupervised Learning of Visual Sense Models for Polysemous Words Kate Saenko Trevor Darrell Deepak.

The Evolving Digital Mathematics Library: A Mathematics Librarian’s Perspective Timothy W. Cole University of Illinois at Urbana-Champaign 8 Dec

A Visualization Model Based on Adjacency Data by Edward Condon Bruce Golden S. Lele S. Raghavan Edward Wasil Presented at Miami Beach INFORMS Conference.

Introduction to LDA Jinyang Gao. Outline Bayesian Analysis Dirichlet Distribution Evolution of Topic Model Gibbs Sampling Intuition Analysis of Parameter.

1 A Web Search Engine-Based Approach to Measure Semantic Similarity between Words Presenter: Guan-Yu Chen IEEE Trans. on Knowledge & Data Engineering,

Data Mining Brandon Leonardo CS157B (Spring 2006).

Object Recognition Part 2 Authors: Kobus Barnard, Pinar Duygulu, Nado de Freitas, and David Forsyth Slides by Rong Zhang CSE 595 – Words and Pictures Presentation.

Image Emotional Semantic Query Based On Color Semantic Description Wei-Ning Wang, Ying-Lin Yu Department of Electronic and Information Engineering, South.

Image Classification for Automatic Annotation

CS 8751 ML & KDDData Clustering1 Clustering Unsupervised learning Generating “classes” Distance/similarity measures Agglomerative methods Divisive methods.

Presented By- Shahina Ferdous, Student ID – , Spring 2010.

Exploiting Ontologies for Automatic Image Annotation Munirathnam Srikanth, Joshua Varner, Mitchell Bowden, Dan Moldovan Language Computer Corporation SIGIR.

Google News Personalization Big Data reading group November 12, 2007 Presented by Babu Pillai.

Clustering Algorithm CS 157B JIA HUANG. Definition Data clustering is a method in which we make cluster of objects that are somehow similar in characteristics.

Towards Total Scene Understanding: Classiﬁcation, Annotation and Segmentation in an Automatic Framework N 工科所錢雅馨 2011/01/16 Li-Jia Li, Richard.

1 CSC 594 Topics in AI – Text Mining and Analytics Fall 2015/16 8. Text Clustering.

2/10/2016Semantic Similarity1 Semantic Similarity Methods in WordNet and Their Application to Information Retrieval on the Web Giannis Varelas Epimenidis.

Enhanced hypertext categorization using hyperlinks Soumen Chakrabarti (IBM Almaden) Byron Dom (IBM Almaden) Piotr Indyk (Stanford)

Correlation between People’s Behaviors in Cyber World and Their Geological Position Lixiong Chen Jan 24 th, 2009.

Instance Discovery and Schema Matching With Applications to Biological Deep Web Data Integration Tantan Liu, Fan Wang, Gagan Agrawal {liut, wangfa,

Nearest Neighbour and Clustering. Nearest Neighbour and clustering Clustering and nearest neighbour prediction technique was one of the oldest techniques.

Clustering Machine Learning Unsupervised Learning K-means Optimization objective Random initialization Determining Number of Clusters Hierarchical Clustering.

Word sense disambiguation with pictures Kobus Barnard, Matthew Johnson presented by Milan Iliev.

A Personal Tour of Machine Learning and Its Applications

Information Organization: Clustering

Matching Words with Pictures

Web Page Cleaning for Web Mining

Matching Words and Pictures

Giannis Varelas Epimenidis Voutsakis Paraskevi Raftopoulou

Find the limit {image} ,024 2,160 -1,

A Review of Researches on Deep Learning in Remote Sensing Application

Presentation transcript:

Clustering Art & Learning the Semantics of Words and Pictures Manigantan Sethuraman

Key Applications Auto Annotation Given image generate associated words. Auto Illustration Given words generate associated images. Sounds Familiar Isnt It ?

Key Ideas Joint Probability Distribution Complete Sense is conveyed by considering words and images together. Hierarchical Model Going from General to Specific. Allowing shared use of information. Providing a search path. Clustering Basically grouping, Images or Regions ?? Soft (Membership is distributed)

Joint Prob. Distr. -> Text Only

Joint Prob. Distr. -> Images Only

Joint Prob. Distr. –> Words & Images

Hierarchical Model Each Node has a probability of generating a word/ image w.r.t the document under consideration. Cluster defines the path. Cluster,Level identifies the node.

Associated Math P(c | d) – Probability of cluster given the document. P(L | c,d) – Probability of the level given the cluster and document. P(i | l,c) – Probability of item given the level and cluster. P(L | c,d) can be roughly represented by their average P(L | c). Model 1 uses the document specific value. Model 2 uses the average value.

Auto Annotation Generate words for a given image Consider the probability of the image belonging to the current cluster. Consider the probability of the items in the image being generated by the nodes at various levels in the path associated to the current cluster. Work the above out for all clusters. We are computing the probability that an image emits a proposed word, given the observed segments, B:

Auto Illustration

Is E-M Used ? E-M is used to train and obtain the hidden information. Clustering Probability of a document d being in the cluster c Image-Word Correlation Probability that Item i of Document d was generated at level L.

Word Sense Disambiguation Semantic Hierarchies Bank -> Financial Institution -> Institution -> Organization. Bank -> slope -> geological formation -> natural object. Word Sense defined by the path to the root. Rather than considering the word as an item, consider the word-sense as an item Six closest words for each occurrence of a word used to disambiguate its sense. For each word the sense which has the largest hypernym (IS_A) sense in common with the neighboring words is chosen.

Questions & Discussion Relationship between Object Recognition paper and this paper… Handling Noise ? Irrelevant descriptions for images Dependence on semantically meaningful segmentation…