1. Imagery-based Digital Collection Retrieval on Web Using Compact Perception Features
Ying Dai
Faculty of software and information science,
Iwate Pref. University
2018/11/28

2. Introduction More and more generation of image information led to
The emergency of automatic index;
The emergency of eliminating the semantic gap between the user and the retrieval system;
The emergency of flexible and intuitive retrieval
2018/11/28

3. Architecture of image retrieval by user intuitive queries (1)
Following the conclusion by B. E. Rogowize [1]
Human observers are very systematic in the evaluation of image similarity, following
Semantic
Color
Structural characteristics
Two most fundamental dimensions describing semantic of images
Human vs. non-human
Natural vs. man-made
Two most fundamental dimensions for humans in judging image similarity:
Human vs. non-human axis: running from the less human-like images of sunsets and clouds, scenes with animals, scenes with small images of humans, to images featuring large full-face portraits of humans.
Natural vs. man-made axis: running from the rock and flower images, nature scenes, nature scenes with man-made objects, to man-made objects with nature, to the building images.
[1] B. E. Rogowitz, et al. “Perceptual image similarity experiments”, Proc. Of SPIE, 1998.
2018/11/28

4. Architecture of image retrieval by user intuitive queries (2)
Architecture of image retrieval which cope with users’ intuitive queries
The system include 5 interfaces.
Category interface perceives the user’s queries regarding semantic to images. Perception word, impression word, and sample interface perceive the users’ queries regarding color and composition to images. The feedback interface perceives user’s individual similarity criteria.
Category database indexes the images by the semantic category, features set DB indexes images by the computation features.
The retrieval modules map the users’ queries to the category or features set database, and retrieve the corresponding images.
The adjusting module map the individual criteria to the decision thresholds, storage the adjusted thresholds, and deliver them to the retrieval modules.
2018/11/28

5. Semantic category of images
Nature with man-made objects
man-made objects
nature scenes
Without beings
With beings
With full-face portraits
Without beings
With beings
With full-face portraits
building, interior, etc.
indoor with humans, etc.
Wild Flower, mountain, sunset, etc.
Wild animal, picnic, etc.
Without beings
with beings
Images can be categorized in the tree structure based on the mentioned conclusion
Flower arrangement, etc.
Amusement grounds, etc.
2018/11/28

6. Cues for image categorization
Annotation
Date
Position
Definition by user
Computation feature set
position: Japan’s geographical data
Annotation:
A image of Japanese garden
definition: garden
Feature set: green, grey nature scene
2018/11/28

7. Extraction of feature set
Unit
Nu.
Signification
Property 3
Means of row sums of SGLD matrices of H, S, V components
Hue and tone distribution
Deviations of row sums of SGLD matrices 5
Eigen values of SGLD matrix of H components
The number of hues
Means of row sums of Eigen SGLD matrix of H components
The principal hues 60
Textural feature parameters of second difference SGLD matrices
Composition-based similarity classification
Here, we use the eigen and difference SGLD matrices to extract the low-level features of images. For the details, the paper presented in IEEE SMC2004 can be referred.
[2] Ying Dai, et al. “Visual perception-based structure analysis of images for digital collection retrieval”, IEEE Proc. Of SMC 2004, CD-ROM, pp , Oct., 2004, Netherlands.
2018/11/28

8. Extraction of objects in images by using
Condition for extracting:
Original image
2018/11/28

9. application of individual criteria in mapping query to features (1)
Objective perception-based color mapping
For the query such as complex hue, red, an so on, By adjusting the threshold set TO regarding n and u, the individual user’s perceptual similarity criteria to the color can be reflected.
2018/11/28

10. application of individual criteria in mapping query to features (2)
Subjective perception-based mapping
2018/11/28

11. application of individual criteria in mapping query to features (3)
Composition similarity mapping
　　　are used to measure the similarity of images in composition by Euclidean distance
2018/11/28

12. Some image retrieval results (1)
Query: retrieving images in the category of papers, fabrics & woods, which are similar to a reference image in structure.
A reference
Retrieved results
2018/11/28

13. Some image retrieval results (2)
Query: retrieving images in the category of paints & pastels, which have multi-hues, appearing calm.
Retrieved results
2018/11/28

14. Some image retrieval results (6)
Query: retrieving images in the category of flowers, which are similar to a reference image in overall color and structure.
A reference
Retrieved results ( 6 best matches)
2018/11/28

15. Satisfaction-based evaluation
Evaluation items:
1) The initial ranking is satisfy; 2) The modified ranking is satisfy;
3) number of images which are near to your imagery for the first ranking of top 10;
4) number of images which are near to your imagery for the modified ranking of top 10;
a), b), c), d) are the average values of positive answers of 1), 2), 3), 4)
Number of images
Image database
Number　of images as query a) b) c) d)
200
Vol. 2 10
77%
86% 6 7
Vol. 9
83%
90%
Vol. 10
80% 8
Vol. 20
2018/11/28

16. Future work & Reference
Improving the Automatic indexing of semantic category
User-based evaluation
Flexibility
Satisfaction
Ease to use
Reference
[1] B. E. Rogowitz, et al. “Perceptual image similarity experiments”, Proc. Of SPIE, 1998.
2018/11/28