1. NATIONAL TECHNICAL UNIVERSITY OF ATHENS Image, Video And Multimedia Systems Laboratory Background http://www.image.ntua.gr

2. NATIONAL TECHNICAL UNIVERSITY OF ATHENS Automatic Image Annotation Input image Automatic segmentationDesired result

3. NATIONAL TECHNICAL UNIVERSITY OF ATHENS  Tool for: Ground truth construction Semi-automatic image annotation  Support of: Automatic segmentation Manual, user driven region merging Export of segmentation masks and textual annotation Image Annotator Tool

4. NATIONAL TECHNICAL UNIVERSITY OF ATHENS Visual Descriptor Ontology  MPEG-7(XML Schema) defines visual descriptors by specifying their components  In VDO (RDFS), descriptors are defined through relations with their components  Descriptors related to higher – level concepts through inference rules  Rules define spatio-temporal constraints

5. NATIONAL TECHNICAL UNIVERSITY OF ATHENS Knowledge- Assisted Analysis Tool Developed in collaboration with CERTH-ITI

6. NATIONAL TECHNICAL UNIVERSITY OF ATHENS KAA Results 0 Sea 0.81172 1 Person 0.948059 2 Sea 0.80658 3 Sand 0.885552 4 Sky 1 …

7. NATIONAL TECHNICAL UNIVERSITY OF ATHENS Approach:  Graph-based representation of images  Semantic vs Syntactic: regions are assigned fuzzy set of labels instead of low-level features  Modification of traditional segmentation algorithms to operate on labelled regions  Simultaneous image segmentation and region labeling Target:  Solve oversegmentation problems  Assign labels with confidence values to regions  Link labels with concepts existing in ontologies Semantic Segmentation

8. NATIONAL TECHNICAL UNIVERSITY OF ATHENS Sea is oversegmented People have been incorrectly merged with the sand RSST segmentation Semantic RSST segmentation Region is assigned to a fuzzy set of labels: {rock/0.89,sand/0.46} Sea segments are merged correctly Semantic Segmentation

9. NATIONAL TECHNICAL UNIVERSITY OF ATHENS Visual Attention & Classification  Generate visual saliency maps  Detect foreground / background  Select most representative regions for classification, based on saliency Lower Classification error OriginalSaliency Map Background Detection