1. ShareCam: Interface, System Architecture, and Implementation of a Collaboratively Controlled Robotic Webcam Dezhen Song (TAMU) and Ken Goldberg (UC Berkeley) IEEE/RSJ International Conference on Intelligent Robots and Systems Finalist for New Technology Foundation (NTF) Award for Entertainment Robots and Systems

2. Teleoperation: Related Work Tesla, 1898 Goertz, ‘54 Mosher, ‘64 Tomovic, ‘69 Salisbury,Bejczy, ‘85 Ballard, ’86 Volz, ’87- Sheridan, ‘92 Sato, ’94 Goldberg, ’94- Presence Journal ‘92- O. Khatib, et al. ’96

3. Internet

8. networked robot:

9. Taxonomy (Tanie, Matsuhira, Chong 00) Multiple Operator, Single Robot (MOSR): Single Operator, Single Robot (SOSR): Single Operator, Multiple Robot (MOSR):

10. Pan, Tilt, Zoom robotic video camera sharecam

11. Entertainment Applications

12. Playing Games

13. Related Work Networked robots –Tanie, K., Chong, N. et al(01) –Jia, S. and K. Takase (01) –Hu, H., Yu, L., Tsui, P., Zhou, Q (01) –Safaric, R. et al. (01) –Goldberg and Siegwart (02) –Coppin, P. and Wagner, M.D. (02) –Konukseven, I., Erkmen, A. et al (02) SOSR –Siegwart, R. and Saucy P. (99) –Paulos, E. and Canny, J. (99) –Tanie, K., Arai, H. et al. (00) –Lynch, K. and Liu, C. (00) –Fong, T., Thorpe, C., et al(01)

14. Related Work SOMR –Hu, Yu, Tsui, Zhou (01) – Jia, Takase (01) MOMR –Fukuda, Xi, Liu, Elhajj et al. (00,02) –Tanie, Chong, et al. (00) MOSR –Cinematrix (91) –Cannon, McDonald, et al. (97) –Goldberg, Chen, et al. (00, 01)

15. Sharecam: System Architecture Users Internet ShareCam Server Video Server

16. Sharecam Software TCP/IP User database Registration Core (with shared memory segments) Apache module Communication Console/Log Login CGI ShareCam web server ShareCam applet InetCam applet Client RS232C HTTP Camera control Calibration Panoramic image generation InetCam server Video server Canon VC- C3 Camera Java Gnu C++ PERL MySQL

17. frame selection problem: given n requests, find optimal frame

18. Problem Definition Requested frames :  i =[x i, y i, z i ], i=1,…,n

19. Problem Definition “Satisfaction” for frame i: 0  S i  1 S i = 0 S i = 1  =    i  =  i

20. Symmetric Difference Intersection-Over-Union Similarity Metrics Nonlinear functions of (x,y)

21. Intersection over Maximum: Requested frame  i, Area= a i Candidate frame  Area = a pipi

22. global satisfaction: for fixed z find  * = arg max S(  )

23. approximation x y d Compute S(x,y) at lattice of sample points: w, h : width and height, g: size range

24. error bound run time: O(n /  3 ) c * Optimal frame Optimal at lattice Smallest frame on lattice that encloses c *

25. Processing Zoom Type Complexity Centralized Discrete Exact O(n 2 ) Centralized Discrete Approx O(nk log(nk)), k=(log(1/ε)/ε) 2 Centralized Contin. Exact O(n 3 ) Centralized Contin. Approx O((n + 1/  3 ) log 2 n) Distributed Discrete Exact O(n), Client: O(n) Distributed Contin. Approx O(n), Client O(1/  3 ) frame selection algorithms

26. ShareCam Application: Game based learning : global environment

30. robotic video cameras motion sensors timed checks sensor network s humans: amateurs and profs. Collaborative Observatories for Natural Environments (CONEs)

31. Frank van der Stappen (CS, Utrecht) Vladlen Koltun (EECS, UC Berkeley) George Bekey (CS, USC) Karl Bohringer (CS, UW) Anatoly Pashkevich (Informatics, Belarus) Judith Donath (Media Lab, MIT) Eric Paulos (Intel Research Lab, Berkeley) Dana Plautz (Intel Research Lab, Oregon) Sariel Har-Peled (CS, UIUC) Thank you. ShareCam: Interface, System Architecture, and Implementation of a Collaboratively Controlled Robotic Webcam

33. Satellite Imaging

35. MIT Press, 2002

36. Networked Robots Tele-Operation Internet Tele-Operation Collaborative Tele-Operation Tele-Actor Co-Opticon Co-Opticon Algorithms www.ken.goldberg.net

38. Infiltrate

44. Networked Robots

45. internet tele-robot:

46. RoboMotes: Gaurav S. Sukhatme, USC

47. Smart Dust: Kris Pister, UCB (Image: Kenn Brown)

48. Networked Cameras

49. Where to look?

50. Sensornet detects activity “Motecams” Other sensors: audio, pressure switches, light beams, IR, etc Generate bounding boxes and motion vectors Transmit to PZT camera Activity localization

51. 1.Network Standards: HTML, Browsers, Java 2.Infrastructure: Backbone, Routers, ADSL 3.Public Adoption 4.Bandwidth: 10 Mbps, 100Mbps, Gbps 5.Video/Audio Compression: MP2,3,4

52. Networked robots Systems that couple communication to control of one or more robots in a network that often includes sensors and remote human operators. Two subclasses : 1) Tele-operated, where human supervisors send commands and receive feedback via the network. Such systems support research, education, and public awareness by making valuable resources accessible to broad audiences. 2) Autonomous, where robots and sensors exchange data via the network. In such systems, the network extends the effective sensing range of individual robots.

53. Challenges: 1.Variable Time Delays, Congestion 2.Latency 3.Access Control, Security, Interfaces 4.Protocol Design 5.Noise, Error Detection and Recovery 6.Deployment, Dynamic Routing 7.Power Management 8.Hybrid Architectures

54. Conventional Security Cameras Immobile or Repetitive Sweep Low resolution

55. Future Work Continuous zoom (m=  ) Multiple outputs: –p cameras –p views from one camera “Temporal” version: fairness –Integrate s i over time: minimize accumulated dissatisfaction for any user Network / Client Variability: load balancing Obstacle Avoidance

56. Outline Collaborative Teleoperation Cinematrix Co-opticon Tele-Twister

58. "In times of terror, when everyone is something of a conspirator, everyone will be in a situation where he has to play detective." -- Walter Benjamin (1938)

59. Statistics of Satellite Imaging 2.5 Billion Market in 2003 Increasing 300% per year since 1999 Major clients –Government / Military –Oil exploration –Weather Prediction –Agriculture Ikonos, 1999

60. Intersection over Maximum: s i ( ,  i ) s i =0.200.210.53 Requested frame  i Candidate frame 

61. Staircase Approximation –Exact algorithm O(n 3/2 log 3 n) [data structure] –Approximation Algorithm O(nk log(nk)), k=(log(1/ε)/ε) 2 Staircase approximation, large constant factors