Virtual New York City: Project Design Capstone Project, Spring 2005 Professor: Zhigang Zhu Students: Peter Giang Robert Hill Edgardo Molina Krishnan Ram Emmanuel Velasco

Background of Virtual NYC  Objective Our project will accomplish two of our goals; to give traffic analysis of New York City and provide users newer, more interesting method of human-computer interaction with New York City. Our program will include a 3-dimensional version of a New York City map that will allow users to navigate through the city using a keyboard and mouse. This gives users an advantage of virtually seeing New York, without having to physically be there.

Background of Virtual NYC  Objective Our project will demonstrate how we can use existing computer vision technologies to improve human- computer interaction. We hope the project will be a basis for many other future projects done by other groups.

Original Augmented NYC Project AugmentedNYC Features AVI Video File Playing Texture Mapping Camera (not shown)

Our Virtual NYC Project VirtualNYC Features Texture Mapping AVI Video File Playing Camera (not shown) 3D Models Hotspots (not yet implemented) Information Panel

AugmentedNYC to VirtualNYC  GUI AugmentedNYC uses the Microsoft Foundation Classes (MFC). We ported AugmentedNYC to VirtualNYC using the QT library. Reasons we decided on QT:  Cross-platform  Easier to code  More readable

AugmentedNYC to VirtualNYC  GUI Information Panel  Info tab contains information about hotspots on the map.  Video tab contains information on all of the videos located on the map.  Editor tab will allow us to add more hotspots and videos.

AugmentedNYC to VirtualNYC  Video We load an AVI file on and display it on the map. Using computer vision techniques, we plan to extract each individual objects (cars) in a video and simulate traffic on the map.

VirtualNYC  3-Dimensional Map 2D to 3D

VirtualNYC  3-Dimensional Map To create a virtual New York City, we place the user in a 3-dimensional world. To give the user the feeling of 3-dimensional, we also add 3D buildings. Eventually we will add texture to the buildings, giving the buildings a more realistic look.

VirtualNYC  Hotspots For landmarks on the map, we have what we call hotspots. Clicking on these hotspots will give more information in the info panel. This feature is currently being developed. Example: Clicking on CCNY displays information on the info panel. Clicking on the URL launches the user’s browser and displays the CCNY website.

Work in Progress  Work on the applications performance Memory usage / texture management Make the 3D rendering faster  Work on the interface Make it useful / intuitive Better Avatars / Icons  Make it run on at least one other platform Main obstacle is Video Libraries  Incorporate video analysis Segmentation and Background subtraction

Future Developments  Satellite/Aerial Imagery (ala Google/Keyhole)  Smarter video processing Live streaming registered feeds Algorithms for Traffic Prediction  GPS information

Future Developments  Real 3D terrain data Captured through LIDAR  3D building / interior / landmark data Captured through laser range scanning systems Create fuller virtual tourism experience