1. Virtual Symphony Orchestra
Object Design Review:
Video Subsystem

2. VSO Object Design Review: Video Subsystem
Agenda
VideoController Interface
Synchronization
Multithreaded video decompression
Bluescreening: background color elimination
Multi Monitor Support
VSO Object Design Review: Video Subsystem

3. VideoController Interface
glContext : NSOpenGLContext
camera : Camera - rootObject : Object3D
+ setOpenGLContext(ctx : NSOpenGLContext)
+ setRootObject(obj : Object3D)
+ setCamera()
+ updateForTimeStamp(ts : CVTimeStamp) : CVReturn
+ setMovieRate(rate : int)
+ setMovieTime(time : TimeRecord)
+ render()
+ startVideo()
+ stopVideo()
VSO Object Design Review: Video Subsystem

4. Initializing the VideoController
The VideoController has to be provided with a OpenGL context and the root object of the orchestra using the setOpenGLContext() and setRootObject() methods
Nodes in the orchestra tree must implement the Object3D protocol
VSO Object Design Review: Video Subsystem

5. VSO Object Design Review: Video Subsystem
Object3D protocol
The VideoData class is provided by the video framework and contains necessary data for rendering.
If no videoData is available the object will not be rendered.
Object3D
+ children() : NSMutableArray
+ hasChildren() : bool
+ visible() : bool
+ position() : Vector3f
+ rotation() : Vector3f
+ videoData() : VideoData
+ height() : float
VSO Object Design Review: Video Subsystem

6. Synchronization / Timestretching
The QuickTime API is used to control the movies
setMovieRate() controls the playback speed of the movies
To avoid running out of sync setMovieTime() can be called in short time intervals
VSO Object Design Review: Video Subsystem

7. Multithreaded video decompression
VideoController
- glContext : NSOpenGLContext - rootObject : Object3D
+ setOpenGLContext()
+ setRootObject()
+ updateForTimeStamp()
+ setMovieRate()
+ setMovieTime()
+ render()
+ startVideo()
+ stopVideo()
VideoDecoder
+ startThread() 1 n
VSO Object Design Review: Video Subsystem

8. Background color elimination
The background elimination of the videos is done using shader programs
Reduces CPU load because these can be processed on the GPU
kernel vec4 greenScreenEffect(sampler image, float lower, float upper) {
vec4 col;
col = unpremultiply(sample(image, samplerCoord(image)));
float delta = col.g-col.r + col.g-col.b;
col.a = smoothstep(lower, upper, delta);
return premultiply(col); }
VSO Object Design Review: Video Subsystem

9. VSO Object Design Review: Video Subsystem
Multi Monitor Support
Multiple display devices can be used by creating multiple VideoController instances and selecting the appropriate display device
The camera object of each controller has to be set in a way that each device shows another display window
Camera
+ position() : Vector3f
+ rotation() : Vector3f
+ viewAngle() : float
+ cameraData() : CameraData
VSO Object Design Review: Video Subsystem

10. Thanks for your attention