Download presentation
Presentation is loading. Please wait.
Published byDomenic Barnett Modified over 9 years ago
1
Image Classification over Visual Tree Jianping Fan Dept of Computer Science UNC-Charlotte, NC 28223 www.cs.uncc.edu/~jfan
2
1. Research Motivation Large-Scale Visual Recognition Inter-concept Visual Correlations rather than independency
3
1. Research Motivation Large-Scale Visual Recognition: Challenges We need to learn large amounts of classifiers for large-scale visual recognition! Some object classes and image concepts are visually-related and hard to be discriminated! Some object classes and image concepts may have huge inner-concept visual diversity!
4
1. Research Motivation Large-Scale Visual Recognition: Challenges Huge inner-concept visual diversity ---simple models may not work, but using complex models may overlap with others! Huge inter-concept visual similarity ---training complexity will increase for distinguishing visually-related concepts! Huge computational cost ---thousands of inter-related classifiers should be trained jointly!
5
2. Collecting Large-Scale Training Images Flickr Images & Other Image Sites Keywords for image crawling
6
Visual Feature Extraction 2. Collecting Large-Scale Training Images
7
Synonymous Concepts: Visual Similarity CVPR2010
8
Synonymous Concepts: Visual Similarity
9
Ambiguous Concept: Visual Diversity 2. Collecting Large-Scale Training Images
10
Ambiguous Concept: Visual Diversity 2. Collecting Large-Scale Training Images
11
Junk Image Filtering subject to: Decision function: R Outliers Majority IEEE Trans. CSVT 2009 2. Collecting Large-Scale Training Images
12
Junk Image Filtering 2. Collecting Large-Scale Training Images
13
Junk Image Filtering 2. Collecting Large-Scale Training Images
14
Most text terms are weakly related or even irrelevant to web images in the same webpage tiger big cat Southeast Asia Russian Chinese Bengal Siberian Indochinese South Chinese …. Noise image
15
Text-Image Alignment for Web Image Indexing WWW2010, PR2014 2. Large-Scale Image Preparation
16
Informative Image Extraction Noise image n Two simple rules: n Aspect ratio (>0.2 or <5) n Image size (min(width, height) > 60 pixel) n Not perfect but produce satisfied results n Unsupervised and computationally efficient
17
Webpage Segmentation Surrounding Text Extraction n Visual-based algorithm n precise but expensive n [Cai et al. MSR-TR’03] n DOM (Document Object Model) based method n computationally efficient
18
Webpage Segmentation Surrounding Text Extraction n Visual-based algorithm n precise but expensive n [Cai et al. MSR-TR’03] n DOM (Document Object Model) based method n computationally efficient
19
Text-Image Alignment for Web Image Indexing WWW2010, PR2014 Near-duplicates share similar semantics!
20
Text-Image Alignment for Web Image Indexing WWW2010, PR2014
21
Duplicate Detection CVPR2012 2. Collecting Large-Scale Training Images Duplicates may mislead classifier training tools!
22
Duplicate Detection 2. Collecting Large-Scale Training Images
23
Automatic Tag-Instance Alignment Missing Tag Prediction ACM MM 2010 CVPR 2012 2. Collecting Large-Scale Training Images
24
3. Visual Concept Network Why we need visual concept network? ---concept ontology, object co-occurrence network, …. Common space: classifier training & concept detection ---visual feature space rather than label space or concept space We need to characterize inter-concept visual correlations rather than others! ACM MM2009 Inter-related learning task determination
25
3. Visual Concept Network
30
Label Tree for Efficient Classification 1, 2, 3, 4 1 1, 3 2, 4 324 Label 1: cat Label 2: mini van Label 3: dog Label 4: fire truck It is a fire truck! Number of dot products needed in the label tree: 1 + 1 = 2 Number of dot product needed in a flat approach: 1 + 1 + 1 + 1 = 4 [Bengio et al. NIPS’2010]
31
Construction of Label Tree SVM 1 Testing Samples SVM 2 SVM N … Confusion Matrix 1, 2, …,10 1,5, 6 2, 4,8, 9 3,7, 10 2, 8 4,9 4 9 Training N one-vs-rest SVMs is very expensive n The SVMs could be unreliable n Huge sample imbalance n Negative samples could mislead the classifier training
32
Visual Similarity Matrix Result is based on ImageNet data set of 1000 categories
33
4. Visual Tree Construction: Hierarchical Clustering Root
34
4. Visual Tree Construction: Hierarchical Clustering
39
Bag-of-Words (BoW)
40
To distinguish visually-similar categories, dictionaries with strong discrimination is critical Joint dictionary learning CVPR 2012 TPAMI2013 5. Joint Dictionary Learning for Discriminative Image Representation
41
7. Large-Scale Classifier Training IEEE Trans. IP 2011, IEEE Trans. PAMI 2014, PR 2013
42
Inference Model Selection for Classifier Training 7. Large-Scale Classifier Training
43
Inference Model Selection for Classifier Training 7. Large-Scale Classifier Training
45
Hierarchical Organization
46
7. Large-Scale Classifier Training Hierarchical Organization
47
7. Large-Scale Classifier Training Hierarchical Organization
48
7. Large-Scale Classifier Training Flat Organization
49
8. Interactive Classifier Assessment VAST 2011
50
8. Interactive Classifier Assessment
52
9. Some Experimental Results
60
10. Conclusions & Current Work A new image representation approach via discriminative dictionary learning; A novel approach for semantic gap quantification; A new solution for inter-related classifier training and large-scale visual recognition. Performing our experiments on large-scale images !
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.