ITCS 6157/8157 Visual Database Fall

Overview Class hour 9:30AM - 12:15PM, Thursday Office hour Thursday 1:30 - 5:00PM Classroom Woodward Hall 130 Instructor - Dr. Jianping Fan - Office – Woodward 205D Webpage Textbook: we will use the slices and papers on the course web page

Why we should have this course? Internet is changing the world Multimedia (especially visual data) is dominating the content of Internet Easy access of multimedia content through Internet could be the future of IT This class will provide training on multimedia content analysis and search!

Why we should have this course? Good job market: Google, Yahoo!.... Have fun: solving real problem Not so “hard” to learn (??) Next generation search engines

How can I access multimedia in database over networks? Networks Course objectives Multimedia Server User Google, Yahoo! & MSN IE

1. How to format multimedia queries? 2. How to represent multimedia content? 3. How to index large-scale multimedia? 4. How to search multimedia in database ? 5. How to transmit query results over IP ? 6. How to control user’s access ? To answer above question, we need to address: Course objectives Everyone has smartphone now

How to build multimedia search engines? How to build text indexing? Text document Natural language processing Inverse File indexingText database Multimedia data multimedia analysis Hash Indexing or others Multimedia database & query Yahoo, Google Can we do multimedia retrieval like Google for text search? Keywords Multimedia ``keywords” Simple extension

Required Techniques How to build multimedia search engines? Computer Vision Technologies for Multimedia Content Analysis Machine Learning Tools for Understanding Multimedia Semantics Database Techniques for Large-Scale Multimedia Indexing Human-Computer Interaction for query formulation, display & exploration

Components from Database System a.Data Representation Schema b.Database Indexing c.Database Storage d.Query Management e.Big Data Analytics

Components from Computer Vision a.Image & Video Analysis & Feature Extraction b.Object Detection & Scene Understanding c.Classifier Training for object and concept detection d.Scene Configuration and Structure

Components from Machine Learning a.GMM & Bayesian Network b.Support Vector Machine (SVM) c.Graphical Models & Structure Learning d.Statistical Inference e.Deep Learning & Big Data Analytics

Database Management System: ITCS6160 or ITCS3160 Computer Vision Machine Learning Programming Skills Willing to work hard If you do not have these background, you should

Course Topics Data Clustering Tools Machine Learning Techniques Multimedia Analysis Technologies Database Indexing Structures Deep Learning & Big Data Analytics Human-Computer Interaction Tools Taking-Home Self-Study Materials Open Discussion & Student Presentation

Grading Composition Project25% Midterm35% Final40% Scale >93% = A 75-93% = B 55-74% = C <55% or cheating = F If you miss 3 classes (three weeks) or more, you are not allowed to take tests (mid-term and final)!

Class Policy You have to attend the class & come to classroom on time (no later than 9:35am) You should be ready to learn from the class: project implementation could be critical You should respect your classmates: come to learn from their presentation!

Classroom Policy No food!!! Drink could be allowed & Cell Phone should be turned off. Small talk is not allowed, but you are welcome to ask questions! Walking inside classroom is not allowed within presentation time!

Course Projects Project implementation project: you need to set up a team or individual to implement one small system for multimedia content analysis or understanding. Paper presentation project: you need to pick one topic to present in the class. MS students are not encouraged to take this kind of project! More information We will offer two kinds of projects:

Implementation Project Develop image/video analysis system using Visual C++ and Java. Each group consists 3-4 students 3-4 hours workload each week is expected Java or C++ assumed Talk to your professor to decide which algorithm you may implement for your project, discuss progress with your professor if necessary Demonstrate your implementation to your professor & get feedback

Paper Presentation Project Present one research topic: you need to talk to your professor to get relevant research papers, prepare presentation slides & present in the class. Well-understanding of the topic Good presentation in the class Be able to answer questions from classmates & professor Topic selection: depending on available topics and professor assignment. If you are PhD student

Course Projects Good grade even you may perform well in final and mid-term tests Practical implementation means more than paper work Good recommendation letter for job hunting: professor can only memorize students with good performance! Research position opportunities If you do wonderful job on course project, you may expect:

Midterm & Final closed books and notes One page notes is permitted Cumulative No makeup Bonus is expected Key components for your final grade If you miss 3 classes or more, you are not allowed to take tests (mid-term and final)!

Suggestions from Instructor Do your best in the class Show your problems to the instructor when you cannot make it Show the evidence to us if you think you are right. Open discussion is welcome, but no small talk

10-hours Golden Rules 3 hours before class: go through the topics, presentation slides and seek some relevant online documents, …; ready to ask questions in class 3 hours in class: listen to domain experts and try to ask questions 4 hours after class: review what you have learnt from the class, do your project and assignments…

Who cares?

Google Search Engine

Who cares?

The way to join them Good grade from class More training on programming skills, especially for multimedia analysis, indexing and retrieval Get recommendation from professor

Recommendation Good grade is very important, but it is not everything! Learning something and solving one problem you like may be more important! Learning from someone who may make you better! Especially your classmates

What areas we will touch? Computer Vision Database Information Retrieval Machine Learning & AI Visualization Networks Statistics & Security

What you may expect Start-up Companies Many wonderful companies & start-ups come from course projects! You could be the next one!

What you may expect Start-up Companies Product search engine for amezon.com, taobao.com Using your smartphones to take pictures, then we will find the cheapest ones for you!

What you may expect Start-up Companies Image Search Engine

What you may expect Start-up Companies Google Glass App: Google glass may change world like i-phone

What you may expect Start-up Companies Digital Camera App: Sony may sale digital cameras with your media organization & search software.

What you may expect Start-up Companies Personal Computer App: IBM Dell may sale PCs with your media organization & search software.

What you may expect Start-up Companies Automatic-Driving Car App: BMW Tesla may sale cars with your object recognition & navigation systems.

What you may expect Start-up Companies Plant Species Identification: Your Kids will be proud of you because you Know every plant species on the world

What you may expect Start-up Companies Plant Species Identification: Your Kids will be proud of you because you Know every plant species on the world

What you may expect Start-up Companies Construction Safety Alarm: educators, government & insurance companies may care

What you may expect Start-up Companies Multimedia Search Engine: Google will definitely care!

What I or UNCC may expect Do not forget to come back UNCC & support our research!

Do your best & have fun! Good students should be able to push your professor to think and work harder not easier!