1. Low latency rendering for mobile augmented reality W. Pasman, A. van der Schaaf, R.L. Lagendijk, F.W. Jansen Delft University of Technology UbiCom – http://www.ubicom.tudelft.nl

2. System sketch

3. Way finding

5. Maintenance, assistance

6. How to do this? 1. System overview 2. Why low latency 3. Low-latency rendering 4. Low-latency tracking 5. Status of project

7. 1. System overview Mobile unit off-loads processes to backbone

8. Camera serves 2 purposes Viewing through camera of another user Tracking the position of the user Mobile unit hardware 17GHz, IR

9. 2. Latency requirements Accurate positioning requires low latency Most demanding situation: head rotation Alignment error depends on lag and rotational speed

10. Humans can detect <8ms But hardly any literature on relation with task. Estimated for mentioned tasks: Max. error 0.5˚ @ 50˚/s latency required ≤ 10 ms

11. VR: several sources of latency Source of latency typical 3D Rendering30-200 ms Tracking15-40 ms Display @50Hz20 ms Fastest reported: 17 (33)ms latency Our goal: 10 ms Rendering + Display8ms Tracking2ms

12. Std. Voodoo 3D game card Render just ahead of raster beam 4 partitions give latency 5-10 ms 3. Low-latency rendering

13. ~ hundred texture-mapped polygons  Simplify virtual world In backbone, using LOD polygon reduction polygon-to-texture techniques Feasible: relatively small #virtual objects

15. Latency-layered system

16. Prototype KURT Realtime Linux to schedule renderer at 200Hz in sync with display Latency measured: 4.2-8.8ms (6.35 avg) Tracker Similar latency-layered techniques as rendering, using inertial tracker in low-latency loop camera feature tracker in outer loop

17. Prototype off-the-shelf components Not yet wireless (tracker & power cables) Rendering system “VRML2” scene graphs changed animation: no 200Hz scene graph traversal -> compile @10Hz to display list @ 200Hz -> tags in displaylist for last-minute updates 5. Status of project

18. Tracking Proto: Intersense IS600 (inertial+acoustic) Working on optical tracking with artificial feducials Later: natural feducials Future Fast & simple hardware for rendering, combining well with video transmission Task - latency relation Applications Incremental scene simplification

19. Conclusion Accurate alignment for AR Latency-layered system architecture First-ever <10ms rendering (no prediction) www.ubicom.tudelft.nl