1. REAL-TIME VIDEO EFFECTS USING A DEPTH SENSOR Robbie Fraser and Ridho Jeftha

2. The Problem Video effects: Difficult Expensive Time consuming

3. The Solution Application: Automated Real time

4. Automation Depth sensor Capture colour with camera (RGBD) Capture depth with sensor (RGBD)

5. Research Question Can image based effects be applied to video footage in real-time?

6. Aim Develop a system to: Capture RGBD from a camera and depth sensor Render a variety of different video effects Test the system: Benchmark performance Evaluate the viability of real-time video effects

7. System Design

8. Mapping System Prepare the data Depth map Point cloud Normal map Refine the data Real time input Reflectance map

9. Video Capture

10. Normal Map Generation Depth Map to Normal Map

11. Normal Map Generation Depth Map to Normal Map

12. Reflectance Map Estimate initial lighting conditions Use normals Estimate reflectance at each point

13. Map Refinement

15. Effects System Four effects: Fog Synthesis Scene Relighting Depth of Field Cartoon Shading

16. Fog Synthesis

17. Scene Relighting

18. Depth of Field

19. Cartoon Shading

20. User Interface Interface with cameras View effects Edit parameters

21. Performance Utilise graphics processors Fragment shaders CUDA

22. Testing Benchmark o Different effects o Different parameters o Different resolutions Performance profiling

23. Success Criterion Performance Entire process at 25fps

24. Work Allocation

25. Thank You Questions?