1. Image-based Techniques for the Creation and Display of Photorealistic Interactive Virtual Environments George Drettakis 1, Maria Roussou 3,4, Nicolas Tsingos 1, Alex Reche 1,2, Emmanuel Gallo 1,2 1 REVES/INRIA, Sophia-Antipolis, France 2 CSTB, France 3 University College London, UK 4 Foundation of the Hellenic World, Hellas (Greece)

2. overall goals n Realistic virtual environments (VE’s) –simplify model creation –enhance realism of display –interactivity n Education and design –enhance learning process –help design and decision making

3. previous approaches n Limited level of realism in VE’s –creation of environments slow, costly and with limited realism –display of environments with low realism n missing shadows and vegetation, flat textures n missing sound n no life (crowds, traffic etc.)

4. previous approaches n Many developments in Computer Graphics, but rarely transferred to VE’s –considered too expensive in resources –engineering aspect overwhelming n Examples of CG techniques –image-based modeling [Debevec 96/98] –point-based rendering [Stamminger 01] –perspective shadow maps [Stamminger 02]

5. overview of our approach n Project context –CREATE IST project n Three components –creating and displaying a realistic VE –interactivity –user-centered design and evaluation

6. data acquisition for CREATE n Two applications areas –cultural heritage and education –urban planning and design n Two corresponding sites –Ancient Messene in Greece (archeology) –Place Massena & Garibaldi in Nice (tramway project)

7. model creation n Image-based modeling –shoot panoramas, stitch and render cube maps

8. resulting model n After calibration, modelling & texturing synthetic image

9. video n Creation of model using Stitcher and ImageModeler (improved versions)

11. view-dependent 3D models n One common model, multiple textures –use blending for view-dependent display n For complex scenes use layering –perform geometric sort to create image layers –use a view-dependent texture per layer

12. view-dependent 3D models

13. Results of Enhanced VE’s

14. urban planning models synthetic image

15. enhanced virtual environment

17. video n Place Garibaldi simulation –shadows –sound

19. interactivity n Cultural heritage –manipulation of architectural members –reconstruction of temple n Urban planning –design and construct “urban features” such as benches or umbrellas

20. user-centered design and evaluation n User task analysis identified expert and novice users: –Cultural heritage (C/H) n experts: archeologists n novice users: general public, children –Urban planning (U/P) n experts: architects n novice users: general public, decision makers

21. user task analysis n Interviews with users, analysis of existing task flow –augment the current tools –add realism and interactivity to enhance productivity n Preliminary studies –cultural heritage with children –preparation for urban case

22. preliminary formative studies n Children in a 3-task experiment (C/H) –7 to 12 years old –CAVE environment n Preparation of usability evaluation with architects (U/P) –direct observation –description of needs –scenario-based evaluation

23. current work n Completion of enhanced VE tools –more realistic trees –integration of crowds and sound n Real-task experiments –architects in a design task for urban planning (under way). Will be evaluated for effectiveness, efficiency, ease of use, and satisfaction.

24. video n Interactivity –design and placement of benches and umbrellas

26. conclusions n Image based techniques –improve realism of VE’s n improved capture & display –combined with interactivity and user- centered design show promise for improved usage of VE’s

27. acknowledgements n CREATE is a 3-year RTD project funded by the EU’s FP5 IST Programme (IST-2001-34231), http://www.cs.ucl.ac.uk/create. The partners of CREATE are University College London (UK), Foundation of the Hellenic World (GR), INRIA (FR), REALVIZ (FR), University of Cyprus (CY), PERCRO (I) and CSTB (FR) n Alias for their donation of Maya n Alexandre Olivier-Mangon for all the image-modeling and virtual modeling, and the videos n Alex Reche and Emmanuel Gallo are supported by CIFRE doctoral fellowships INRIA-CSTB

28. thank you http://www-sop.inria.fr/reves http://www.cstb.fr http://www.cs.ucl.ac.uk/create