1. Project Implementation for ITCS4122
Jianping Fan
Department of Computer Science
University of North Carolina at Charlotte

2. You can select one of them
Semantic image classification & Visual Analytics
Image Clustering & Visual Analytics
Deep Image Segmentation & Visual Analytics
Interactive Data Analytics

3. 1. Build a system for supporting semantic image classification & visual analytics
---you are encouraged to use deep learning for feature extraction
System Components:
a. Image analysis and feature extraction
(1) Image segmentation using JSEG
You can download jseg image segmentation source code from:
(2) Feature extraction: extract color histogram, color moments,
Tamular textures (moments et al.) from each
homogeneous regions
(3) Region classifier training
You can download software from:

4. (4) Region merging
After the labels for the homogeneous regions are obtained, the neighboring regions
with the same label will be merged as one big region for the relevant salient object
(5) Interesting point detection
You can download software from:
Detecting the object via SIFT features
(6) Object-Based feature extraction
Extract color histogram, color moments, textures, and SIFT from salient objects

5. (7) Concept classifier training
positive samples
negative samples
positive support vectors
margin
negative support vectors
You can download software from:

6. Deep Networks such as AlexNet, VGG, GoogleNet, ResNet, …
(8) Visualization for features and classification results
(9) User-system interaction for result evaluation and integrating human advises for
further data analytics.
Deep Learning approach for this project
Outputs: Features & List of Object Classes & Concepts
Visualization of Category-driven Data Distributions
Deep Networks such as AlexNet, VGG, GoogleNet, ResNet, …
Input Images
Parameter Fine-Tune

7. You need to build classifiers for:
Salient Objects
Tree, grass, red flower, yellow flower, sky
water, sand, road, car, building
Image Concepts
Garden, beach

8. 2. Automatic Image Clustering & Visual Analytics
---you are encouraged to use deep learning for feature extraction
You can download 1,000,000 images from
You can extract one of the following visual features: color histogram or SIFT features, you can also try CNN features, you can get source code from
You can apply AP clustering technique to get an optimal partition of these 1,000,000 images according to their visual similarity, you can get AP clustering source code from

9. Image Query Result Clustering
You can go to Google Images and type in keywords such as ``apple”, ``bank”, ``cloud” et al
Download the returned images (more than 1000 images), total images will be 100,000.
Extract color histogram or SIFT by using
Apply AP clustering to get an optimal partition of the returned images

10. Deep Networks such as AlexNet, VGG, GoogleNet, ResNet, …
Visualization for features and classification results
User-system interaction for result evaluation and integrating human advises for
further data analytics.
---you are encouraged to use deep learning for feature extraction
Outputs: Features & Image Clusters
Visualization of Data Distributions
Deep Networks such as AlexNet, VGG, GoogleNet, ResNet, …
Input Images
Parameter Fine-Tune

11. 3. Deep Image Segmentation & Analytics
You can use DeepLab v3, U-nets, Mask RCNN,… to do semantic image segmentation
Integrating visualization to evaluate the segmentation results
Leveraging human advises to improve deep image segmentation

12. Project Requirements
Project implementation: you need to spend time to understand the project and the relevant source code and compile these source codes and make them work in your computer and try to make some revisions.
Penalties: TA will test your system by using his images, so it is easy for us to check whether you really do the project or not. If we find that you did not spend enough time to work on your project, you will receive big penalties on your project grade.

13. 4. Traditional Tools for Interactive Data Visualization
Input Data
Feature Extraction & Data Clustering
Data & Knowledge Visualization
User Assessment & Feedbacks
Constraint-Driven Data Clustering

14. Group or not Each group may have no more than 5 students
We encourage individual project & bonus is expected for individual project
Bonus will be given to good implementation such as improving on algorithms or including nice interface & visualization