FYP Project LYU0303: 1 Video Object Tracking and Replacement for Post TV Production

Augmented Reality  Augmented Reality (AR) is a form of human- machine interaction where information is presented in the file of view of an individual augmenting the real world scene necessitating that the user maintains a sense of presence in that world. augmenting the real world scene necessitating that the user maintains a sense of presence in that world. the virtual images are merged with the real view to create the augmented display. the virtual images are merged with the real view to create the augmented display. technology for merging the real and virtual image streams in tight constrain. technology for merging the real and virtual image streams in tight constrain.

AR (movie)

AR Animation (movie)

Key Technologies of AR  accurate registration of the virtual objects with the real world image camera position, viewing parameters camera position, viewing parameters digital composition, scene lighting, object texture mapping digital composition, scene lighting, object texture mapping real-time processing constrain real-time processing constrain user viewing method user viewing method

AR Systems  Monitor Based Augmented Reality  Video See-through Augmented Reality Display  Optical See-through Augmented Reality Display

AR using Monitor (movie)

AR using Head Mount Device (movie)

ARCADE: Augmented Reality Computing Arena for Digital Entertainment  AR provides new ways of digital entertainment possibilities when combining with traditional entertainment channels.  Key applications: TV and Movie productions combining fantasy with reality

LYU0303: Video Object Tracking and Replacement for Post TV production  Design the special pattern markers for 3D objects to be applied on objects during the video shooting  Develop efficient algorithm to calculate the object’s positions and orientations in the video Non real-time algorithm Non real-time algorithm MPEG-1 or DV quality video MPEG-1 or DV quality video The special pattern marks and the detection algorithm must survive from the partial occlusion of the object. The special pattern marks and the detection algorithm must survive from the partial occlusion of the object.

LYU0303: Video Object Tracking and Replacement for Post TV production  Develop algorithm to determine the lighting condition of the scene in a video  Generate realistic images to replace the video object according to it’s positions and orientations non real-time algorithm non real-time algorithm MPEG-1 or DV quality video MPEG-1 or DV quality video Simple object, soda can or cereal box Simple object, soda can or cereal box The algorithm must survive from the partial occlusion of the object The algorithm must survive from the partial occlusion of the object Photometric registration problem Photometric registration problem

Popular 2D Markers

Illustration of possible Special Pattern Markers for 3D Object

Same Object Two Brand Names

References  International Symposium on Mixed and Augmented Reality, ISMAR Can be downloaded from IEEE website using Campus network (IP address restrictions) Can be downloaded from IEEE website using Campus network (IP address restrictions) ieeexplore.ieee.org/xpl/tocresult.jsp?isNumber=24594 ieeexplore.ieee.org/xpl/tocresult.jsp?isNumber=24594  Augmented Reality Website  MagicBook  Virtual Showcase