1. Exact and Distributed Algorithms for Collaborative Camera Control Dezhen Song * A. Frank van der Stappen † Ken Goldberg * * UC Berkeley, USA † Utrecht University, Netherlands

2. Dezhen Song

3. Webcams

5. n users 1 pan, tilt, zoom robotic camera

6. “ShareCam”

7. Example input: 7 requested frames:

8. One Optimal Frame ShareCam Problem: Given n requests, find optimal frame

9. Taxonomy (Tanie, Matsuhira, Chong 00) Multiple Operator, Single Robot (MOSR): Single Operator, Single Robot (SOSR): Single Operator, Multiple Robot (MOSR):

10. Related Work MOSR systems –Cinematrix (91) –Cannon, McDonald, et al. (97) –Goldberg, Chen, et al. (00, 01) –Goldberg, song, et al. (02) Internet robots –Tanie, K., Chong, N. Et al(01) –Jia, S. And K. Takase (01) –Hu, H., Yu, L., Tsui, P., Zhou, Q (01) –Safaric, R. Et al. (01) –Goldberg and Siegwart (02)

11. Related Work Facilities Location Problems –Megiddo and Supowit [84] –Eppstein [97] –Halperin et al. [02] Rectangle Fitting –Grossi and Italiano [99,00] –Agarwal and Erickson [99] –Mount et al [96] –Kapelio et al [95]

12. Related Work Similarity Measures –Kavraki [98] –Broder et al [98, 00] –Veltkamp and Hagedoorn [00] Distributed robot algorithms –Sagawa et al [01], Safaric[01] –Parker[02], Bulter et al. [01] –Mumolo et al [00], Hayes et al [01] –Agassounon et al [01], Chen [99]

13. Problem Definition Requested frames :  i =[x i, y i, z i ], i=1,…,n

14. Problem Definition Assumptions –Camera has fixed aspect ratio: 4 x 3 –Candidate frame  = [x, y, z] t –(x, y)  R 2 (continuous set) – z  Z (discrete set) (x, y) 3z 4z

15. Problem Definition “Satisfaction” for user i: 0  S i  1 S i = 0 S i = 1  =    i  =  i

16. Symmetric Difference Intersection-Over-Union Satisfaction Metrics Nonlinear functions of (x,y)

17. Intersection over Maximum: Requested frame  i, Area= a i Candidate frame  Area = a Satisfaction Metrics pipi

18. Intersection over Maximum: s i ( ,  i ) s i =0.200.210.53 Requested frame  i Candidate frame 

19. Requested frame  i Candidate frame  (x,y) (for fixed z)

20. Satisfaction Function – s i (x,y) is a plateau One top plane Four side planes Quadratic surfaces at corners Critical boundaries: 4 horizontal, 4 vertical

21. Objective Function Global Satisfaction: for fixed z ShareCam problem: Find  * = arg max S(  )

22. S(x,y) is non-differentiable, non-convex, but piecewise linear along axis-parallel lines. Properties of Global Satisfaction

23. ShareCam Algorithms Bruteforce Algorithm –Compute S at each pixel (x,y) –O(whmn): w, h: width and height of panoramic image m: number of zoom levels n: # users

24. Approximation Algorithm x y d Compute S(x,y) at lattice of sample points:

25. Approximation Algorithm –Run Time: –O(w h m n / d 2 )  * : Optimal frame : Optimal at lattice : Smallest frame at lattice that encloses  *

26. Exact Algorithm Define “Virtual corners” –Consider a pair of requested frames and –Their critical boundaries y x

27. Exact Algorithm Virtual corner: Intersection between boundaries –Self intersection: –Frame intersection : y x

28. Exact Algorithm Claim: An optimal point occurs at a virtual corner. Proof: –Along vertical boundary, S(y) is a 1D piecewise linear function: extrema must occur at x boundaries

29. Exact Algorithm Exact Algorithm: Check all virtual corners  (mn 2 ) virtual corners  (n) time to evaluate S for each  (mn 3 ) total runtime

30. Improved Exact Algorithm Sweep horizontally: solve at each vertical –Sort critical points along y axis: O(n log n) –1D problem at each vertical boundary O(nm) –O(n) 1D problems –O(mn 2 ) total runtime O(n) 1D problems

31. Distributed Algorithm More users  More computers available

32. Distributed Algorithm At the Server –Sort horiz. boundaries –O(n log n) At the Client –Solve 1D problem for own vertical boundaries. –O(nm) O(n(m+ log n)) Total Four 1D problems

33. Examples

35. www.tele-actor.netwww.tele-actor.net / sharecam

36. New Approx Algorithms: –With Har-Peled, Koltun –Stair-like approximation –Dynamic segment tree –O(n log n) Weighted Requests Current Work

37. Future Work Continuous zoom (m=  ) Multiple outputs: –p cameras –p views from one camera “Temporal” version: fairness –Integrate s i over time: minimize accumulated dissatisfaction for any user Network / Client Variability: load balancing Obstacle Avoidance

38. The Tele-Actor Operators Server

39. Summary Satisfaction Metric: Intersection over Maximum ShareCam Problem : find  * = arg max S(  ) Critical Points at Virtual Corners Exact Algorithms: Distributed Algorithm: tele-actor.net/sharecam O(mn 2 ) O(mn)

41. Summary A collaborative camera control system Satisfaction metric Virtual corner based algorithms Distributed algorithm www.tele-actor.net/sharecam

42. Introduction Regular web-camCollaborative camera control Queue

44. Internet Interface:

45. Results & Discussion Speed of naive (B) and fast (V): –AMD K7 950Mhz –1.2 GB memory –JDK 1.3.1 –For a fixed z