1. Virtual Light Field Group vlfproject@cs.ucl.ac.uk University College London GR/R13685/01 Research funded by: Introduction: Basic Structures of the Virtual Light Field Mel Slater Department of Computer Science University College London 4 th June, 2004 http://www.cs.ucl.ac.uk/vlf

2. VLF Project Introduction Problem Background – Light Field Fundamental data structures & algorithm Implementation Issues Early Results Conclusions

3. VLF Project Problem: Global Illumination in Real-Time The goal of this research real-time rendering of globally illuminated scenes – Interaction between any type of surface properties – Technically – to provide a fast if approximate solution of the rendering equation. – L(p,  ) = L e (p,  ) +  f(p,  i,  ) L(p,  i ) cos  i d  i

4. VLF Project Two Solutions Today Virtual Light Field (VLF) – Global illumination for mixtures of diffuse and specular surfaces – Real-time walkthrough implemented – Dynamic changes possible – not implemented Virtual Light Field for Ray Tracing (VLF-RT) – Partial global illumination for dynamically changing scenes – Specular-Specular light transport only

5. VLF Project Global Illumination Ray tracing – highly view dependent, models only specular reflection Path tracing – Monte Carlo solution to rendering equation (view dependent) Radiosity – view independent, models only diffuse reflection Photon mapping/Monte Carlo – rays through light emitters (view independent) view dependent ray tracing phase

6. VLF Project Faster Global Illumination? Caching – reuse elements of a G.I. for one view in other views. Ward, G. and Simmons, M. (1999) The Holodeck Ray Cache: An Interactive Rendering System for Global Illumination in Nondiffuse Environments, ACM TOG, 18(4):361-98. Precompute – e.g., estimate point lights that give similar solution to global illumination. Walter, B., Alppay, G., Lafortune, E.P.F., Fernandez, S. and Greenberg, D. P. (1997) Fitting virtual lights for non-diffuse walkthroughs, SIGGRAPH 97, 45–48. H/W ray tracing – implement BSP ray tracing in hardware efficient way. Wald, I., Slusallek, P., Benthin, C., Wagner, M.: Interactive Rendering with Coherent Ray Tracing, Eurographics 2001 Proceedings.

7. VLF Project Exploitation of Light Fields LF provides a ‘brute force’ solution to the radiance equation. Ideal for use with digital pictures of real scenes. http://graphics.stanford.edu/projects/lightfield/

8. VLF Project Restrict L to 4D The domain of L can be restricted to 4D L(s,t,u,v) can represent all rays between two parallel planes – a ‘light slab’. st plane uv plane Here radiance is flowing in direction uv to st. camera positions

9. VLF Project Light Field = 6 Light Slabs 6 light slabs can represent rays in all directions: – Near-far, Far-near – Left-right, Right-left – Bottom-top, Top-bottom Discretisation provides approximation to ‘all rays’.

10. VLF Project Assigning Radiance to Each Ray st plane uv plane cop Image plane Place the cop at each grid point on st plane Use uv plane as an image plane. Each ray is assigned a radiance.

11. VLF Project Synthesising a New Image st plane uv plane Image plane cop New image plane s0s0 s1s1 s2s2 s3s3 s4s4 u0u0 u1u1 u2u2 u3u3 u4u4 The image ray is approximated By L(s 3, u 1 )

12. VLF Project Synthetic Scenes Can be used for synthetic scenes – If OpenGL used, not much point! – Real point is for global illumination. Gortler used term ‘virtual light field’ to refer to light field for a synthetic scene. We extend the term to mean a LF for a virtual scene – But without the use of a separate rendering system – With illumination computed within the LF itself – Which illuminates whole scenes not single objects.

13. VLF Project Virtual Light Field Any ray is a point in 4D space R 4 Suppose ‘all rays’ can (somehow) be represented. Consider one ray:- t1 t2 0 1 t3 t4

14. VLF Project Rendering Objects into the Field Object ‘rendered’ into the 4D space – induces a segmentation of each intersecting ray Associated with each ray is a vector of T-Intersections: – T-Intersection object id t-value direction (‘left’ or ‘right’) unshot radiance and radiance (=0 initially)

15. VLF Project Original Scene of Objects All ray-object intersections are pre-computed

16. VLF Project Now Switch on the Lights Switch on each light emitting object Light distributes through the environment

17. VLF Project Now Switch on the Lights Light distributes through the environment Switch on each light emitting object

18. VLF Project Now Switch on the Lights Light distributes through the environment Switch on each light emitting object

19. VLF Project Need Discrete Representation Just a description of how light distributes through an environment Need a method that is computable in reasonable time – requires a discrete representation of the virtual light field

20. VLF Project Required Properties Suppose R 4 (n) a representation of R 4 Limit property – R 4 (n)  R 4 assymptotically with n Indexable – r n  R 4 (n) then associated vector of T-Intersections must be found in near constant time – r  R 4 then its closest r n  R 4 (n) must be found in constant time Set of ‘all rays’ = Union of Parallel Subfields

21. VLF Project Spherical Angle Representation Base is rotated through ‘all’ spherical angles base z y x   Spherical angles vary ‘uniformly’ cover the hemisphere

22. VLF Project Scene Covered Each box is one parallel subfield – a parallel subfield has all rays parallel total light field = set of all parallel subfields (i,j, ,  ) Vertical rays

23. VLF Project Use Ray Approximations Approximate each ‘true’ ray by its nearest ray in the collection of all rays

24. VLF Project Use Ray Approximations Perfect specular reflection – use closest approximate ray to the true reflected ray Diffuse reflection – use all rays through the intersection point

25. VLF Project Use Ray Approximations Perfect specular reflection

26. VLF Project Use Ray Approximations Perfect specular reflection – use closest approximate ray to the true reflected ray

27. VLF Project Use Ray Approximations Perfect specular reflection – use closest approximate ray to the true reflected ray Diffuse reflection – use all rays through the intersection point

28. VLF Project Rendering into R 4 (n) is Fast Vertical subfield rendering is equivalent to rasterisation Any other angle result can be obtained by rotation – Standard hardware can be used geometry (rotation) raster-scan + z-buffer z y x

29. VLF Project Process of Distributing Light Cycle 0 – Emit from sources mark any object hit as in receipt of unshot radiance Cycle 1…n – Each marked object compute all intersecting rays – reflect out all rays carrying unshot radiance mark hit objects Repeat until convergence

30. VLF Project Diffuse Reflection Does not cause combinatorial explosion Each diffuse object can be done in two passes – a pass = compute all rays through object Pass 1 – accumulate incoming radiance (gathering) Pass 2 – distribute radiance (shooting)

31. VLF Project Viewing Put a lens and image plane into the light field – (can throw the object specs away) ‘Render the lens’ to compute all rays through it – like rendering one polygon many times Refract rays through lens to hit image plane Time is constant - independent of original number of objects

32. VLF Project Costs A naïve implementation requires huge memory to store all the rays (~ 1Gb minimum). This is with resolution of about 100  100 with 6000 rotations No exploitation of coherence

33. VLF Project Low Quality Images – but signs of global Illumination

34. VLF Project Tiling Approach – Exploit Coherence Two major changes – Order of propagation – Coherence amongst rays (tiling) Large saving in memory For each object{ for each PSF{ propagate(); } For each PSF{ for each object propagate(); }

35. VLF Project Tiling The N  N resolution of the PSF is divided into square tiles of resolution M  M (M<N) PSF

36. VLF Project Tiling PSF The N  N resolution of the PSF is divided into square tiles of resolution M  M (M<N) Each tile maintains a list of object representations:

37. VLF Project Each tile maintains a list of object representations: { u_short object_id; EdgeTable visibility_mask; Image radiance; Image unshot_radiance; } Tiling Objects are sampled at full PSF resolution Images may be sampled at less than full resolution PSF The N  N resolution of the PSF is divided into square tiles of resolution M  M (M<N)

38. VLF Project PSF (S) tile Propagation run through each tile in each PSF, propagating and storing radiance for objects in the tiles

39. VLF Project Propagation run through each tile in each PSF, propagating and storing radiance for objects in the tiles PSF (S) tile

40. VLF Project Propagation run through each tile in each PSF, propagating and storing radiance for objects in the tiles PSF (S) tile

41. VLF Project Propagation run through each tile in each PSF, propagating and storing radiance for objects in the tiles PSF (S) tile

42. VLF Project PSF (R) For specular – compute new PSF and store unshot radiance for same surface in respective tile. PSF (S) tile

43. VLF Project PSF (S) tile PSF (R) For diffuse – accumulate radiance surface map for distribution at next iteration.

44. VLF Project For diffuse – accumulate radiance surface map for distribution at next iteration. PSF (S) tile PSF (R) For specular – accumulate radiance in unshot and total radiance maps.

45. VLF Project Propagation … No need to keep surfaces sorted within a tile Construct an ‘exchange buffer’ on the fly – For each surface can use z-buffer to compute – which other surfaces it can exchange energy with along a tile.

46. VLF Project ‘Uniform’ Subdivisions on a Hemisphere Major issue is to get ‘uniform’ partition of a hemisphere into ‘data points’ p=(x,y,z) – These correspond to the PSF directions Problem – given any query point on the hemisphere find the nearest data point in (fast!) constant time.

47. VLF Project Standard Method Subdivide a regular polyhedron that approximates the hemisphere: (0,0,1) (1,0,0) (0,1,0) Level 0

48. VLF Project Standard Method (0,0,1) (1,0,0) (0,1,0) Level 1

49. VLF Project Standard Method (0,0,1) (1,0,0) (0,1,0) Level 2

50. VLF Project Subdivision Method Each subdivision works on vertices projected to the sphere:

51. VLF Project Subdivision Method Each subdivision works on vertices projected to the sphere:

52. VLF Project Subdivision Method Each subdivision works on vertices projected to the sphere:

53. VLF Project Subdivision Method Each subdivision works on vertices projected to the sphere:

54. VLF Project Query Method Previous query methods used recursive subdivision to search for the nearest data point to a query point. Alternative image based method renders a quadrant of the hemisphere, with each small region around a data point in a different ‘colour’ – ‘Colour’ is a look-up index into an array of data points (and associated information).

55. VLF Project Any query point q is projected using the same camera parameters as in the original rendering of the quadrant, and the colour of the image at that point is used to look up the associated data point. Slater, M. (2002) Constant Time Queries on Uniformly Distributed Points on a Hemisphere, Journal of Graphics Tools, 7(1):33-44.

56. VLF Project Summary and Terms Total set of rays (VLF) = – Union of parallel subfields Each a rotation of a canonical PSF Each PSF subdivided into tiles – Tile is a tube that intersects the scene Tile – Maintains list of surfaces it intersects

57. VLF Project Surface represented by Fragment that intersects the tile A diffuse radiance map (DRM) (diffuse / surface) A total radiance map (TRM) (directional / tile) – Accumulation of all energy from the surface in the direction of the corresponding PSF An unshot radiance map (URM) – Energy that is accumulated but has yet to be distributed

58. VLF Project Three Phases Rasterisation – Computes the PSFs and tiling – a 2D rasterisation operation Energy propagation – Distributes energy starting from the light sources Rendering – Finds radiance values that intersect an image plane from a COP.

59. VLF Project Implementation The new method was used with larger numbers of rays. Higher quality images could be produced Diffuse surfaces only Diffuse and specular

60. VLF Project Significant Problems The results showed that global illumination was being achieved But images too noisy, inaccurate, … EPSRC project funded…