1 A Systems Architecture for Ubiquitous Video Neil J. McCurdy and William G. Griswold Mobisys, 2005 Presented by Sangjae Lee

2 One-line Comment  Authors address that  ubiquitous video systems are essential in the future  How to get to build these system?  Abstractions of the infinite cameras  The introduction of virtual space concept  How to adopt in ubiquitous environments?

3 Ubiquitous Video (I) “the walls have eyes” wireless networked video cameras Ubiquitous Video It is inevitable in future However, we do not have to wait for the future Ubiquitous video streams using today’s technology

4 Ubiquitous Video (II)  Entering dangerous, restricted or remote sites with head-mounted cameras  Commanders can navigate through the remote environment  Example scenarios  Police Special Weapons and Tactics (SWAT) teams  Hazardous materials (HazMat)  Police monitoring

5 Statement of the Problem  To design ubiquitous video systems managing the incoming streams  It is challenging  wild condition  Live, real-time access, Uncalibrated cameras, Lighting conditions and etc  A naïve approach  The video on an array of monitors  Ideal solution  Infinite cameras in the field  Allow the user to move seamlessly

6 Solution Approach  Practical Solution  Illusion of the ideal system  Operating under the constraints imposed by the real environment  RealityFlythrough  abstraction  Stitching the multiple video streams together into a single scene  Non-trivial to construct  The limited number of cameras  Mobility (position, orient)

7 RealityFlythrough - Abstraction

8 RealityFlythrough - The virtual cameras  Cameras project their images onto a virtual wall

9 RealityFlythrough

10 System overview  How might such a system be built?  We need  Cameras  image capture component  Location sensors  sensor capture component  Stream combine  Need to be combine sensor data to the appropriate frame  Multipoint Control Unit  RealFlythrough Engine

11 RFT Engine (I)  Deciding which images to display at any point in time

12 RFT Engine (II)  Still Image Generator  producing and managing the still-images that are generated from the live camera feeds  Transition Planner  Determining the path that will be taken to the desired destination  Choosing the images that will be displayed along that path  Transition Executer  Actually moves the user along the chosen path  Camera repository  The store for all known cameras

13 Design pattern (environment state)  Environment stat model (virtual cameras) Open arrow : inheritance Open diamonds : a reference Filled-in diamonds : ownership

14 Design pattern (view) Virtual camera may need to be rendered by multiple cameras Alpha blend

15 The birdseye view

16 Ubiquitous environments  Consider a typical case  The user wish to move to a live camera  A naïve approach  Determine the location and orientation of the live camera  Compute optimal trajectory to get to the target  Determine the images to be shown along the path  Ubiquitous video environment  The destination camera may change its position/orientation when the plan was computed/executed  Wrong destination  The path may not be the optimal ones It does not work !!!!

17 Ubiquitous environments - A dynamic path  A dynamic path  The destination is now a moving target  The transition planner can look ahead some interval  Determine the best image to display at that time

18 Still Image Generation  Key to the success of the infinite camera abstraction  The presence of sufficient cameras  To handle this problem  Take snapshot of the live video  Generate additional cameras from these  Represents still-images  Static images source  The use of still-imagery  Help achieve the abstraction of infinite camera coverage  Imprecise  Option  Never see older images  Older image look different (sepia tone)

19 Evaluation (Effectiveness of the Abstraction)  experiment at the campus food court  Too many images were being presented  Disorientation  GPS accuracy was very low  After changing these problems  adjustments  Reduce image overload, too much movement, location accuracy filtering  A positive comments by users  Let’s try one  That was pretty nice  It’s pretty accurate  That was kind of cool

20 Evaluation (System Performance)  Bottleneck on the server

21 Future Modifications  Better High Level Abstraction  Sound  Scale to Multiple views with Multiple Servers

22 Conclusion  harness ubiquitous video is designed  With few live cameras  providing the abstraction of infinite camera coverage  Virtual camera is introduced  Still-images were automatically captured  Dynamic path is used

23 Critique (I)  Strong Points:  Their applications are very fresh  Ubiquitous Video  Authors design an whole system  The system consists of several components  Address relationship between these components  Also, they defined several problems itself and propose solution for it  Problems due to ubiquitous environment  They implemented this system and experimented in real world  It is very difficult to run system on real world.

24 Critique (II)  Weak Points  Experimental Measurement is poor  Needs to gathering data about the comments and use statistical views  No consideration about  The abilities of mobile device  Too ideal case  Camera resolution  Computational power  The bandwidth of wireless network  RTF’s outputs are a little bit mess and dirty.  Available/unavailable image  Overlapped images  Server’s bottleneck is very serious to apply the industry

25 Critique (III)  New Idea  A few fixed camera will be better performed  If there is some of fixed or slightly moving camera, abstraction will be better  If the number of cameras is increased, the performance increase  Scalability problems  Distributed servers  One of the problems is a bottleneck on server  Let be the server with distributed manner  a core server, gathering outputs from distributed servers  Separate the render from transition planner  Then, we should consider about the bandwidth of wireless network seriously  Video transmission on ubiquitous environment  Lower batteries, lower computation.