Frank Eliassen - Lucas Luiz Provensi - Roman Vitenberg Lille,

World Opera Gilgamesh Production Scenarios – Location tracking – Motion recognition – Light Control – Multimedia sensor networks – Information services Challenges

 Worldwide distributed operas ◦ Singers that are remotely located can interact as if they are in the same stage. ◦ Musicians distributed around the world can play together. ◦ Audience can see the opera from one of the stages or from other remote place.

First World Opera Production (2012) 3 Stages: Tromsø – Stockholm – Copenhagen; Based on The Epic Of Gilgamesh, an epic poem from old Mesopotamia.

 Virtual Stages Uruk Gilgamesh Cedar Forest Humbaba Land of the Dead Enkidu

 The number of multimedia flows/streams coming in and going out will be different for each stage  It can change at runtime as well  At certain times, distributed actors may be interacting with each other while the musicians at all stages keep playing ◦ Actor has a fixed assignment to a physical stage but may travel between virtual stages ◦ A physical avatar to represent the actor at the others stages, like a puppet with a display in the face that shows the video stream of the face of the actor

 Musicians are very sensitive to audio quality but do not care too much about video quality  Remote musicians are sensitive to maestro acting. They are not very tolerant to delay in the maestro's video streaming  When acting against distributed peers, actors are not tolerant to delay and they need a good video quality and enough resolution to see face details  Actors also need to hear audio, e.g., when in a synchronized dialog ◦ Embed a speaker into the avatar so that the actors have the sensation that the voice is coming from that point

 The maestro will be at the city stage, and he will represent the sun god and will be walking around the stage The conductor location will be tracked to physically reproduce the movement of the maestro at the other stages.

 Position of actors in a virtual world.

Wii Mote (Nintendo) Composed by accelerometer and optical sensor. Can be extended with a gyroscope.

Project Natal (Microsoft) Depth sensor consisting of an infrared projector combined with a monochrome CMOS sensor

 Replicate musicians movements on virtual entities or phenomena.

 Replicate actors movements on virtual avatars

 Lighting sensors can provide information lighting condition to scene controller software.

 Multimedia sensor networks ◦ New sources of media content : views of different angles and audio collected at different points.

 Services offered to the audience at the stages and outside. ◦ Information about the ongoing act; ◦ Access to alternative media streams; ◦ Shared comments with other users. Gilgamesh Enkidu Bull Act 2 Gilgamesh and Enkidu fights the Bull of Heaven

 Questions?