1. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE RP3 Middleware & TennisSense Gregory O’Hare, Richard Tynan, Conor Muldoon & Anthony Schoofs

2. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE TennisSense Overview Application Goals: 1. Sense tennis player 2. Assist tennis coach/player to improve their game Technical Goals: 1. Correlate multiple sensory data sources 2. Automated analysis of sensory data 3. Delivery of data to the coach/player

3. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Initial Configuration Camera x 9 Ubisense x 4Inertial x 100(approx)Foster-Miller x 4 ? Image Analysis Player location Ball location Rally Start... XML Annotation Player zone Player location... Query: Is player serving?

4. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Why Use Middleware? Enhances interoperability between systems Facilitates data management – e.g. fusion/correlation Future Applications Flexibility – adapt to new parts of the application automatically Reuse – automatically reuse application components Data structure and data semantics can be encoded

5. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Proposed Configuration Camera x 9 Ubisense x 4Inertial x 100(approx)Foster-Miller x 4 Image Analysis XML Annotation Query: Is player serving? Multi-Agent System Middleware XML Annotation Image Analysis PZT Camera Control QoS Metric: how often is player in frame?

6. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Pros and Cons Abstraction to data sources, controllers and actuators Consistent and homogenous communication protocol ACL – FIPA compliant messages Real time decision making Facilitates addition of new sub-applications Agent migration can effectively balance load May possibly slow down analysis

7. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Proposed Scenario Base Line Rally Detection 1. Sense player and ball data 2. Identify signatures in data e.g. base line rally 3. PZT control of cameras QoS – optimise duration player is in shot Agents may refine standard PZT values for the cameras based on a QoS feedback loop

8. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Completed Tasks 4 files Richie – Ubisense data -> agent beliefs Anthony – 3 camera files -> agent beliefs Using intermediate/indirect file based approach not appropriate Awaiting feedback from DCU on the proposed scenario Transitioning to using live data rather than logs

9. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Current Configuration Camera x 9 Ubisense x 4Inertial x 100(approx)Foster-Miller x 4 ? Image Analysis Player location Ball location Rally Start... XML Annotation Player zone Player location... Query: Is player serving? BEL(playerZone(5)) BEL(playerLocation(7, 10)) …

10. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Goal: PTZ Camera Control Set pan, tilt and zoom to ensure player is in frame Some cameras virtually pan and tilt Some cameras will physically pan and tilt Achieved through data integration Full Picture

11. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Agent Software Ubisense data indicates the player’s coordinates - (x(t), y(t), z(t)) Perceptor takes live data feed of player’s location Beliefs are created e.g. BEL(playerLocation(x, y, z)) Actions can be triggered according to the agent’s desires For example, controlling PZT camera values to ensure player is in shot

12. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE PTZ Calculation

13. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Camera Control Agent actuator invokes the http API of the networked video camera Example: http://camera/ptz.cgi? parameter=value Absolute positioning: http://camera/ptz.cgi?pan=20&tilt=-10&zoom=900 http://camera/ptz.cgi?pan=20&tilt=-10&zoom=900 Alternative with areazoom: http://camera/ptz.cgi?areazoom=30,40,300 http://camera/ptz.cgi?areazoom=30,40,300 Can also perform relative positioning with a URL

14. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Current Configuration 2 Camera x 9 Ubisense x 4 BEL(playerLocation(7, 10)) …

15. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Remaining Tasks 1a. Middleware closer to sensors 1b. Viability - in network processing/real time experiments 2. Video analysis to automatically determine % of time player is covered by camera 3. Close the loop – provide the control/feedback loop for cameras and ubisense data 4. Live Testing

16. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE This is a slide title This is a first level heading And this is a second level heading Here is another first level heading Followed by another second level And one more level 1 And a final second level So these slides are good for about 4 main bullets And some detail text per slide

17. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE This is a 2Col text slide Level 1 heading Followed by level 2 Level 1 heading Followed by level 2 Level 1 heading Followed by level 2 Level 1 heading Followed by level 2 Level 1 heading Followed by level 2 Level 1 heading Followed by level 2 Level 1 heading Followed by level 2 Level 1 heading Followed by level 2

18. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE Half-way down the Vallee Blanche Note the blue glacial ice! Neat eh?

19. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE A right hand picture slide Emerald Bay, Lake Tahoe The most amazing place! Level 1 heading Level 2 heading Level 1 heading Level 2 heading Level 1 heading Level 2 heading

20. UNIVERSITY COLLEGE DUBLIN  DUBLIN CITY UNIVERSITY  TYNDALL NATIONAL INSTITUTE A right hand chart slide Level 1 heading Level 2 heading Level 1 heading Level 2 heading Level 1 heading Level 2 heading Level 1 heading Level 2 heading