© Fraunhofer IGD Immersive Media and 3D Internet – Future Challenges and Solutions Fraunhofer IGD Consultation Meeting Networked Media Systems Brussels, 19 Januar 2010 Fraunhofer-Institut für Graphische Datenverarbeitung IGD Fraunhoferstraße Darmstadt

© 2009 Fraunhofer IGD Immersive Media Technologies at Fraunhofer IGD Scalable immersive display systems Heyewall: 35MPixel 3D Powerwall Omniwall: 360 o Vision based interaction Multitouch / gesture based / multimodal interaction Implicit user interaction (attention recognition, user classification) Instant Reality Framework Scenegraph based framework for AR/VR, Clustering, Scalable Photorealistic rendering from embedded to HPC

© 2009 Fraunhofer IGD No Diffusion of 3D content into every day’s life (behind expectation) Missing developments of core 3D internet technology Missing enabler for an easy, standardised access, retrieval and distribution of 2D/3D content (dynamic information space) Lack of creative and intelligent access tools still not a single technology widely used, though high rendering quality and network performance at hand most approaches were heavily technology driven, no clue about cognitive-perceptual aspects ! Wrong methodology in the past ! overloaded environments and standards, e.g. X3D, Second Live… isolated solutions and lack of psycho-cognitive research ! State of Play – What IS the 3D Internet ?

© 2009 Fraunhofer IGD “3D Internet4All” diffused in daily life, through single point of access technology (e.g. 3D content prosumption into browser architecture) “Hybrid Media Internet” coexistence of 2D/3D worlds within the MR continuum being creative and intelligent access point for a “High Fidelity Internet” Multi-World Coexistence, ensure world transition without loss of information, privacy, coherent and adaptive interaction possibilities a single access technology widely used, accounting for sustainable “Quality on the move Standardised 3D content and 2D/3D information models User centric design and no black box developments Secure, safe and preserving attributes such as “free spirits” Socialising and Participatory Spaces, allow users to take part/immerse State of Play – What MIGHT a 3D Internet be ?

© 2009 Fraunhofer IGD Future Challenges of Networked Media Dynamic content adaptation over different distribution platform Harmonisation of metadata models  Facilitate interoperability between different standards to retrieve media documents in a common way Effective recommendation systems  help users to cope with the tremendous amount of available data Semantic analysis of media content to automatically generate metadata of high quality Aid to preserve, learn about and communicate our cultural heritage by content based media enhancement or 3D object digitization technologies a layer to guarantee privacy and grow protection within new eBusiness models and tools Confluence of media New coding and compression models; Network profiling, content modeling, prediction models, 3D progressive transmission schemes, dynamic P2P

© 2009 Fraunhofer IGD Future Challenges of Immersive Media Enhanced Multimodality enabling intentional and behavioural interaction via different input channels (VR/AR/MR) Confluence of media channels, its carrier and distribution in ever growing dynamic information spaces Behavioural Modelling, Action Driven User Interfaces Work towards an integrated 3D virtualisation pipeline, from real-time 3D scene and object acquisition (capturing reality), modelling and transmission to visualisation of seamless photo-realistic 3D objects in Mixed Reality environments Simulated Reality as new mantra for future networked immersion being at the crossroads of mathematical modelling, sensor fusion, computer science and cognitive/perceptual computing