1. 3D Computer Vision and Video Computing Introduction Instructor: Zhigang Zhu zzhu@ccny.cuny.edu CSc 83300 Spring 2006 Reading in 3D Computer Vision and Video Computing Lecture 1: Introduction http://www-cs.engr.ccny.cuny.edu/~zhu/GC-Spring2006/CSc83300-Video-Computing-2006.html

2. 3D Computer Vision and Video Computing Research at CcvcL n Research Projects at l The City College Visual Computing Lab u 3D video mosaicing, representation and rendering u Multimodal human signature detection u Robotized Sensor networks u 3D cargo and vehicle inspection l The NSF PI Meeting Talk The NSF PI Meeting Talk l The UTRC Hi-Tech Meeting Talk The UTRC Hi-Tech Meeting Talk C C V C L http://www-cs.engr.ccny.cuny.edu/~zhu/research.html

3. 3D Computer Vision and Video Computing Purpose-Driven Reading n A slightly new approach: purpose-driven reading l Goals come first l Readings come second l Projects come last n Some more details l Get to know what you want to do l Figure out potential projects and the goals l Find related papers to read and present l Try out something new and/or interesting l Write out something u Ideally publications for a conference or journal

4. 3D Computer Vision and Video Computing Homework # 1 n Submit your resume with l Education (where/what/how good) l Skills (what you are good at: GUI, algorithm, hardware) l Work Experiences ( R&D) l Publications (if any) l Projects related to imaging, vision and graphics (IVG) u Your advisors (if any) l Courses taken u Particularly math, CS, EE related to IVG u Algorithms, Signal processing, etc

5. 3D Computer Vision and Video Computing Course Organization n Lectures by me (4 lectures) l Basics in 3D computer vision l Camera models, calibration, stereo, motion n Talks by others l CCNY lecture series, GC CS Colloquium n Project Ideas by us l 3D vision, video and robotics l Some ongoing work at the CCNY Visual Computing Lab l Your research interests (2 nd level up) n Presentations by you ( ~ 3 times each, 2 – 3 each class meet) l Others’ work (1~2) l Your work proposals and work reports (1~2) n Office Hours l Tuesday 4:30 – 6:00 pm

6. 3D Computer Vision and Video Computing Course Web Page n Lectures available in Powerpoint format n Reading schedule will be posted on the web n All assignments will be distributed over the web n Additional materials and pointers to other web sites http://www-cs.engr.ccny.cuny.edu/~zhu/GC-Spring2006/CSc83300-Video-Computing-2006.html March 6-10: A talk by Prof. Harvey Rhody @ RIT Monday or Tuesday ?

7. 3D Computer Vision and Video Computing Book n Textbook l “Introductory Techniques for 3-D Computer Vision” Trucco and Verri, 1998 n Additional readings when necessary l “Computer Vision – A Modern Approach” Forsyth and Ponce, 2003 l “Three-Dimensional Computer Vision: A Geometric Viewpoint” O. Faugeras, 1998 n On-Line References and Reading Materials

8. 3D Computer Vision and Video Computing C++ and Matlab n C++ l For some simple computation, you may use C++ n Matlab l An interactive environment for numerical computation l Available on Computer Labs machines (both Unix and Windows) u Matlab primer available on line (web page) u Pointers to on-line manuals also available l Good rapid prototyping environment n You should use C++ and/or Matlab for your homework assignments and project(s); Java will also be fine

9. 3D Computer Vision and Video Computing Grading n Homework (about 2~3): 20% n Course Work: 40% l Reading – Related work by others l Experiments – Results by yourselves l Written Report – Papers or TRs n Presentations (2~3 per student): 40%

10. 3D Computer Vision and Video Computing 3D Computer Vision n What makes (3D) Computer Vision interesting ? l Image Modeling/Analysis/Interpretation u Interpretation is an Artificial Intelligence Problem F Sources of Knowledge in Vision F Levels of Abstraction u Interpretation often goes from 2D images to 3D structures F since we live in a 3D world l Image Rendering/Synthesis/Composition u Image Rendering is a Computer Graphics problem u Rendering is from 3D model to 2D images 2D images 3D world CVCG

11. 3D Computer Vision and Video Computing IP vs CV n Image processing (mainly in 2D) l Image to Image transformations l Image to Description transformations l Image Analysis - extracting quantitative information from images: u Size of a tumor u distance between objects u facial expression l Image restoration. Try to undo damage u needs a model of how the damage was made l Image enhancement. Try to improve the quality of an image l Image compression. How to convey the most amount of information with the least amount of data

12. 3D Computer Vision and Video Computing Video Computing n Some Examples on Video Computing l Video compression l Video surveillance l Video manipulation (video texture, video composition) l Video mosaicing l Video segmentation l 3D video

13. 3D Computer Vision and Video Computing Approaches n Three interesting approaches: l Computational Vision:ImageStructure u David Marr (MIT) l Knowledge-Based Vision:ImageStructure u Active Vision l Applied VisionImagesFunction(Control) u many others n Different methodological assumptions n Different methods n Different results n Where is Video Computing? l an example.... draw your own conclusions! general specific

14. 3D Computer Vision and Video Computing Related Fields n Image Processing: image to image n Computer Vision: Image to model n Computer Graphics: model to image n Pattern Recognition: image to class l image data mining/ video mining n Artificial Intelligence: machine smarts l Machine perception n Photogrammetry: camera geometry, 3D reconstruction n Medical Imaging: CAT, MRI, 3D reconstruction (2 nd meaning) n Video Coding: encoding/decoding, compression, transmission n Physics & Mathematics: basics n Neuroscience: wetware to concept n Computer Science: programming tools and skills? All three are interrelated! AI Applications basics

15. 3D Computer Vision and Video Computing Applications n Visual Inspection (*) n Robotics (*) n Intelligent Image Tools n Image Compression (MPEG 1/2/4/7) n Document Analysis (OCR) n Image Libraries (DL) n Virtual Environment Construction (*) n Environment (*) n Media and Entertainment n Medicine n Astronomy n Law Enforcement (*) l surveillance, security n Traffic and Transportation (*) n Tele-Conferencing and e-Learning (*)

16. 3D Computer Vision and Video Computing Job Markets n Homeland Security l Port security – cargo inspection, human ID, biometrics l Facility security – Embassy, Power plant, bank l Surveillance – military or civil n Media Production l Cartoon / movie/ TVs/ photography l Multimedia communication, video conferencing n Research in image, vision, graphics, virtual reality l 2D image processing l 3D modeling, virtual walk-through n Consumer/ Medical Industries l Video cameras, Camcorders, Video phone l Medical imaging 2D -> 3D

17. 3D Computer Vision and Video Computing Research at CcvcL n Research Projects at l The City College Visual Computing Lab u 3D video mosaicing, representation and rendering u Multimodal human signature detection u Robotized Sensor networks u 3D cargo and vehicle inspection l The NSF PI Meeting Talk The NSF PI Meeting Talk l The UTRC Hi-Tech Meeting Talk The UTRC Hi-Tech Meeting Talk C C V C L http://www-cs.engr.ccny.cuny.edu/~zhu/research.html

18. 3D Computer Vision and Video Computing Research at CcvcL n Research Projects at l The City College Visual Computing Lab u 3D video mosaicing, representation and rendering u Multimodal human signature detection u Robotized Sensor networks u 3D cargo and vehicle inspection l The NSF PI Meeting Talk The NSF PI Meeting Talk l The UTRC Hi-Tech Meeting Talk The UTRC Hi-Tech Meeting Talk C C V C L http://www-cs.engr.ccny.cuny.edu/~zhu/research.html