1. ITCS 6157/8157: Visual Database
Jianping Fan
Fall 2017

2. Overview Class hour 6:30PM - 9:15PM, Wednesday
Office hour Wednesday 1:30 - 5:00PM
Classroom Woodward Hall 130
Instructor - Dr. Jianping Fan -
Office – Woodward 205D
Webpage
Textbook: we will use the slices and papers on the course web page

3. Why we should have this course?
Internet & smartphones are changing the world

4. Why we should have this course?
Multimedia (especially visual data) is dominating the content of Internet

5. Why we should have this course?
Easy access of multimedia content through Internet & mobile devices could be the future of IT

6. Why we should have this course?
This class will help you make these happen!
Image/video analysis for feature extraction
Database indexing for fast search
Machine learning for content understanding
Many others…..

7. Why we should have this course?
Good job market: Google, Facebook....
Have fun: solving real problem
Not so “hard” to learn (??)
Next generation search engines

8. Course objectives Networks Google, Yahoo! & MSN IE User
How can I access multimedia in database over networks?
Networks
User
Multimedia Server

9. Course objectives 1. How to format multimedia queries?
To answer above question, we need to address:
1. How to format multimedia queries?
2. How to represent multimedia content?
3. How to index large-scale multimedia?
4. How to search multimedia in database ?
5. How to transmit query results over IP ?
6. How to control user’s access ?
Everyone has smartphone now

10. Can we do multimedia retrieval like Google for text search?
How to build multimedia search engines?
Yahoo, Google
How to build text indexing?
Natural language processing
Text document
Keywords
Text database
Inverse File indexing
Simple extension
multimedia analysis
Multimedia data
Multimedia ``keywords”
Multimedia database & query
Hash Indexing or others

11. Required Techniques
How to build multimedia search engines?
Computer Vision Technologies for Multimedia Content Analysis
Machine Learning Tools for Understanding Multimedia Semantics
Database Techniques for Large-Scale Multimedia Indexing
Human-Computer Interaction for query formulation, display & exploration

12. Components from Database System
Data Representation Schema
Database Indexing
Database Storage
Query Management
Big Data Analytics

13. Database Indexing Structures

14. Database Indexing Structures

15. Components from Computer Vision
Image & Video Analysis & Feature Extraction
Object Detection & Scene Understanding
Classifier Training for object and concept detection
Scene Configuration and Structure

16. Computer Vision

17. Components from Machine Learning
GMM & Bayesian Network
Support Vector Machine (SVM)
Graphical Models & Structure Learning
Statistical Inference
Deep Learning & Big Data Analytics

18. Machine Learning

19. Database Management System: ITCS6160 or ITCS3160 Computer Vision
Pre-Requirements of this Class
Database Management System: ITCS6160 or ITCS3160
Computer Vision
Machine Learning
Programming Skills
Willing to work hard
If you do not have these background, you should

20. Course Topics Data Clustering Tools Machine Learning Techniques
Multimedia Analysis Technologies
Database Indexing Structures
Deep Learning & Big Data Analytics
Human-Computer Interaction Tools
Taking-Home Self-Study Materials
Open Discussion & Student Presentation 3

21. Grading Composition Scale Project 25% (excellent implementation can be
up to 45%)
Midterm & Final 75%
Scale
>93% = A
75-93% = B
55-74% = C
<55% or cheating = F
If you miss 3 classes (three weeks) or more, you are
not allowed to take tests (mid-term and final)! 3

22. Class Policy
You have to attend the class & come to classroom on time (no later than 6:35pm)
You should be ready to learn from the class: project implementation could be critical
You should respect your classmates: come to learn from their presentation!

23. Classroom Policy
No food!!! Drink could be allowed & Cell Phone should be turned off.
Small talk is not allowed, but you are welcome to ask questions!
Walking inside classroom is not allowed within presentation time!

24. Course Projects
We will offer two kinds of projects:
Project implementation project: you need to set up a team or individual to implement one small system for multimedia content analysis or understanding.
Paper presentation project: you need to pick one topic to present in the class. MS students are not encouraged to take this kind of project!
More information 3

25. Implementation Project
Develop image/video analysis system using Visual C++ and Java.
Each group consists 3-4 students
3-4 hours workload each week is expected
Java or C++ assumed
Talk to your professor to decide which algorithm you may implement for your project, discuss progress with your professor if necessary
Demonstrate your implementation to your professor & get feedback 3

26. Paper Presentation Project
If you are PhD student
Present one research topic: you need to talk to your professor to get relevant research papers, prepare presentation slides & present in the class.
Well-understanding of the topic
Good presentation in the class
Be able to answer questions from classmates & professor
Topic selection: depending on available topics and professor assignment. 3

27. Course Projects
If you do wonderful job on course project, you may expect:
Good grade even you may perform well in final and mid-term tests
Practical implementation means more than paper work
Good recommendation letter for job hunting: professor can only memorize students with good performance!
Research position opportunities 3

28. Midterm & Final closed books and notes Cumulative No makeup
One page notes is permitted
Cumulative
No makeup
Bonus is expected
Key components for your final grade
If you miss 3 classes or more, you are not allowed to
take tests (mid-term and final)! 3

29. Suggestions from Instructor
Do your best in the class
Show your problems to the instructor when you cannot make it
Show the evidence to us if you think you are right.
Open discussion is welcome, but no small talk

30. 10-hours Golden Rules
3 hours before class: go through the topics, presentation slides and seek some relevant online documents, …; ready to ask questions in class
3 hours in class: listen to domain experts and try to ask questions
4 hours after class: review what you have learnt from the class, do your project and assignments…

31. Who cares?

32. Who cares?
Google Search Engine
Google Search Engine

33. Who cares?

34. The way to join them Good grade from class
More training on programming skills, especially for multimedia analysis, indexing and retrieval
Get recommendation from professor

35. Recommendation Good grade is very important, but it is not everything!
Learning something and solving one problem you like may be more important!
Learning from someone who may make you better! Especially your classmates

36. What areas we will touch?
Computer Vision
Database
Information Retrieval
Machine Learning & AI
Visualization
Networks
Statistics & Security

37. Start-up Companies What you may expect
Many wonderful companies & start-ups come from course projects!
You could be the next one!

38. Start-up Companies What you may expect Product search engine for
amezon.com, taobao.com
Using your smartphones to take pictures, then we will
find the cheapest ones for you!

39. What you may expect
Start-up Companies
Image Search Engine

40. Start-up Companies What you may expect Google Glass App: Google
glass may change world like
i-phone

41. Start-up Companies What you may expect Digital Camera App: Sony
may sale digital cameras with
your media organization & search
software.

42. Start-up Companies What you may expect Personal Computer App: IBM
Dell may sale PCs with your media
organization & search software.

43. Start-up Companies What you may expect Automatic-Driving Car App: BMW
Tesla may sale cars with your object
recognition & navigation systems.

44. Start-up Companies What you may expect
Plant Species Identification: Your Kids will be proud of you because you Know every plant species on the world

45. Start-up Companies What you may expect
Plant Species Identification: Your Kids will be proud of you because you Know every plant species on the world

46. Start-up Companies What you may expect
Construction Safety Alarm: educators, government & insurance companies may care

47. Start-up Companies What you may expect Multimedia Search Engine:
Google will definitely care!

48. What I or UNCC may expect
Do not forget to come back UNCC
& support our research!

49. why not ask "stupid" questions?
Do your best & have fun!
Good students should be able to push your professor
to think and work harder not easier!

50. I am a nice professor if you do your jobs!