Project Implementation for ITCS4122 Jianping Fan Department of Computer Science University of North Carolina at Charlotte http://www.cs.uncc.edu/~jfan

You can select one of them Semantic image classification & Visual Analytics Image Clustering & Visual Analytics Deep Image Segmentation & Visual Analytics Interactive Data Analytics

1. Build a system for supporting semantic image classification & visual analytics ---you are encouraged to use deep learning for feature extraction System Components: a. Image analysis and feature extraction (1) Image segmentation using JSEG You can download jseg image segmentation source code from: http://vision.ece.ucsb.edu/segmentation/jseg/ (2) Feature extraction: extract color histogram, color moments, Tamular textures (moments et al.) from each homogeneous regions (3) Region classifier training You can download software from: http://www.cs.cornell.edu/people/tj/svm_light/ http://www.csie.ntu.edu.tw/~cjlin/libsvm/

(4) Region merging After the labels for the homogeneous regions are obtained, the neighboring regions with the same label will be merged as one big region for the relevant salient object (5) Interesting point detection You can download software from: http://vision.ucla.edu/~vedaldi/code/sift/sift.html Detecting the object via SIFT features (6) Object-Based feature extraction Extract color histogram, color moments, textures, and SIFT from salient objects

(7) Concept classifier training positive samples negative samples positive support vectors margin negative support vectors You can download software from: http://www.cs.cornell.edu/people/tj/svm_light/ http://www.csie.ntu.edu.tw/~cjlin/libsvm/

Deep Networks such as AlexNet, VGG, GoogleNet, ResNet, … (8) Visualization for features and classification results (9) User-system interaction for result evaluation and integrating human advises for further data analytics. Deep Learning approach for this project Outputs: Features & List of Object Classes & Concepts Visualization of Category-driven Data Distributions Deep Networks such as AlexNet, VGG, GoogleNet, ResNet, … Input Images Parameter Fine-Tune

You need to build classifiers for: Salient Objects Tree, grass, red flower, yellow flower, sky water, sand, road, car, building Image Concepts Garden, beach

2. Automatic Image Clustering & Visual Analytics ---you are encouraged to use deep learning for feature extraction You can download 1,000,000 images from You can extract one of the following visual features: color histogram or SIFT features, you can also try CNN features, you can get source code from http://clickdamage.com/sourcecode/index.php You can apply AP clustering technique to get an optimal partition of these 1,000,000 images according to their visual similarity, you can get AP clustering source code from http://www.image-net.org/ http://bonsai.hgc.jp/~mdehoon/software/cluster/software.htm

Image Query Result Clustering You can go to Google Images and type in keywords such as ``apple”, ``bank”, ``cloud” et al Download the returned images (more than 1000 images), total images will be 100,000. Extract color histogram or SIFT by using http://clickdamage.com/sourcecode/index.php Apply AP clustering to get an optimal partition of the returned images http://bonsai.hgc.jp/~mdehoon/software/cluster/software.htm

Deep Networks such as AlexNet, VGG, GoogleNet, ResNet, … Visualization for features and classification results User-system interaction for result evaluation and integrating human advises for further data analytics. ---you are encouraged to use deep learning for feature extraction Outputs: Features & Image Clusters Visualization of Data Distributions Deep Networks such as AlexNet, VGG, GoogleNet, ResNet, … Input Images Parameter Fine-Tune

3. Deep Image Segmentation & Analytics You can use DeepLab v3, U-nets, Mask RCNN,… to do semantic image segmentation Integrating visualization to evaluate the segmentation results Leveraging human advises to improve deep image segmentation

Project Requirements Project implementation: you need to spend time to understand the project and the relevant source code and compile these source codes and make them work in your computer and try to make some revisions. Penalties: TA will test your system by using his images, so it is easy for us to check whether you really do the project or not. If we find that you did not spend enough time to work on your project, you will receive big penalties on your project grade.

4. Traditional Tools for Interactive Data Visualization Input Data Feature Extraction & Data Clustering Data & Knowledge Visualization User Assessment & Feedbacks Constraint-Driven Data Clustering

Group or not Each group may have no more than 5 students We encourage individual project & bonus is expected for individual project Bonus will be given to good implementation such as improving on algorithms or including nice interface & visualization