Download presentation
1
Image Classification for Automatic Annotation
Jianping Fan Department of Computer Science University of North Carolina at Charlotte
2
Feature Extraction Object-Based Approach Visual Features Input Image
Salient Objects Color histogram, Tamura texture, Locations ……. Visual Features
3
Feature Extraction Image-Based Approach
4
Image Representation Images Salient Objects Color Color histogram
Tamura Texture Wavelet Texture histogram Shape Interest Point set
5
Feature-Based Image Representation
dimension Feature dimension Feature dimension Curse of Dimensionality
6
Feature-Based Image Representation
Tree-Based Database Indexing
7
Feature-Based Image Representation
Tree-Based Database Indexing Nearest Neighbor Search Overlapping between Different nodes What’s the solution?
8
Feature-Based Image Representation
Where idea for tree-based indexing come from? Library: 12,000,000 books
9
Feature-Based Image Representation
Books in Library Natural Sciences Social Sciences Computer Science Electrical Engineering Dancing Computer Languages Researches I get it! Database Multimedia Too easy! 11!
10
Feature-Based Image Representation
What is the solution? Let's go back to pinciple! Concept Hierarchy or Ontology
11
Hierarchical Concept Organization
Concept Ontology
12
Hierarchical Concept Organization
Concept Ontology
13
6. Classifier Training for Atomic Image Concepts
Beach Atomic Image Concept Different Patterns of Co-Appearances of Salient Objects Water & Sky Water & Sand Water, Sand, Sky, … Feature Subset 9 Feature Subset 1 Feature Subset 2 Feature Subsets
14
6. Classifier Training for Atomic Image Concepts
Curse of Dimensionality # samples needed increase with # dimensions (generally exponentially) . Human labeling is expensive Some features are redundant Proposal Joint SVM boosting and feature selection
15
Subspace 1 Subspace 2 Subspace 3 … Subspace N Higher Level Classifier
Boosting SVM Classifier Training PCA PCA PCA PCA High dimensional feature space Subspace Subspace Subspace 3 … Subspace N SVM SVM SVM SVM Low-level classifiers … Weak Classifier 1 Weak Classifier 2 Weak Classifier 3 Weak Classifier N Boosting for optimal combination Higher Level Classifier High-level classifier Less training samples due to dimension reduction Reuse training results on low-level concepts More selection opportunities compared to filter and wrapper
16
6. Classifier Training for Atomic Image Concepts
Kernel-Based Data Warping Kernel Function:
17
6. Classifier Training for Atomic Image Concepts
Kernel for Color Histogram Statistical Image Similarity Kernel
18
6. Classifier Training for Atomic Image Concepts
Wavelet Filter Bank Kernel
19
6. Classifier Training for Atomic Image Concepts
Wavelet Filter Bank Kernel
20
6. Classifier Training for Atomic Image Concepts
Interest Point Matching Kernel
21
6. Classifier Training for Atomic Image Concepts
Interest Point Matching Kernel
22
6. Classifier Training for Atomic Image Concepts
Multiple Kernel Learning SVM Image Classifier
23
6. Classifier Training for Atomic Image Concepts
Dual Problem Subject to:
24
6. Classifier Training for Atomic Image Concepts
25
6. Classifier Training for Atomic Image Concepts
Some Results Beach Scene Garden Scene
26
High-Level Image Concept Modeling
Inter-Concept Similarity Modeling Nature Scene Flower View Garden Beach Nature Scene: Larger Hypothesis Space & Large Variations of Visual Properties! Garden, Beach, Flower view: Different but share common visual properties!
27
7. Classifier Training for High-Level Image Concepts
Challenging Problems Error Transmission Problems Training Cost Issue Knowledge Transferability and Task Relatedness Exploitation
28
7. Classifier Training for High-Level Image Concepts
Error Transmission Problem The classifiers for low-level image concepts cannot recover the errors for the classifiers of high-level image concepts!
29
7. Classifier Training for High-Level Image Concepts
Error Transmission Problem Outdoor Flower View Garden Beach Errors for the classifiers of atomic image concepts may be transmitted to the classifiers for the high-level image concepts!
30
7. Classifier Training for High-Level Image Concepts
Training Cost Issue Multiple Hypotheses outdoor garden flower view beach Large Diversity of Contents
31
7. Classifier Training for High-Level Image Concepts
Knowledge Transferability & Task Relatedness Exploitation Outdoor Flower View Garden Beach They are different but strongly related!
32
7. Classifier Training for High-Level Image Concepts
Multi-Task Learning Which tasks are strongly related? How to quantify the task relatedness? How to integrate such task relatedness for training large-scale related image classifiers?
33
7. Classifier Training for High-Level Image Concepts
Related Learning Tasks Nature Scene Flower View Garden Beach They are different but strongly related! Concept Ontology can provide a good environment for multi-task learning!
34
7. Classifier Training for High-Level Image Concepts
Related Learning Tasks
35
7. Classifier Training for High-Level Image Concepts
Relatedness Modelling outdoor garden flower view beach : Common Prediction Structure
36
7. Classifier Training for High-Level Image Concepts
Joint Objective Function Subject to:
37
7. Classifier Training for High-Level Image Concepts
Dual Problem Subject to:
38
7. Classifier Training for High-Level Image Concepts
Biased Classifier Training Dual Problem Subject to:
39
7. Classifier Training for High-Level Image Concepts
Common Prediction Structure Nature Scene Flower View Garden Beach Common Visual Properties
40
7. Classifier Training for High-Level Image Concepts
Hierarchical Boosting
41
7. Classifier Training for High-Level Image Concepts
Biased Classifier for Parent Node
42
7. Classifier Training for High-Level Image Concepts
Hierarchical Boosting to Generate Classifier for Parent Node
43
7. Classifier Training for High-Level Image Concepts
Performance Evaluation
44
7. Classifier Training for High-Level Image Concepts
Performance Evaluation
45
7. Classifier Training for High-Level Image Concepts
Advantages of Hierarchical Boosting Handling inter-concept similarity via multi-task learning Reducing training cost Enhancing discrimination power of the classifiers
46
8. Hierarchical Image Classification
Overall Probability Parent Node Path 1 Path 2 Path C Children Node 1 Children Node 2 Children Node C
47
8. Hierarchical Image Classification
Some Results
48
10. Query Result Evaluation
Allow Users to See Global View!
49
10. Query Result Evaluation
Allow Users to See Similarity Direction!
50
10. Query Result Evaluation
Allow Users to Zoom into Images of Interest!
51
10. Query Result Evaluation: Red Flower
Allow Users to Select Query Example Interactively!
52
10. Query Result Evaluation: Sunset
Allow Users to Look for Particular Images!
53
11. Training Image Observation
54
11. Training Image Observation
55
11. Training Image Observation
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.