1. Image Space Rendering and Texturing ©Celine Loscos 2004 Anthony Steed 1999

2. Introduction We are interested in the process of displaying the scene in the drawing window In other words, we are interested in choosing a colour for each pixel depending on how each polygon is viewed from the viewpoint

3. Introduction One way is with ray tracing: –trace a ray from each pixel, and intersect it with the polygons of the scene –Problem: This is expensive We are looking for methods that project the polygons on the window, and then decisions on the pixel colour are made in the image space

4. Overview Coping with depth Gouraud shading Phong shading Texturing

5. Coping with depth When all polygons are projected on the screen, they may overlap The problem is to know which one is in front of the others There are several methods to treat this –Scan-line depth buffering –Z-buffer –Recursive sub-division –Trade-offs Z stands for the depth

6. Scan-line depth buffering: Extending the AET (active edge table) Easy enough if polygon do NOT intersect Put all polygon edges into ET with extra depth information and proceed as before except … … now consider overlapping ranges of edges

7. Representation of the edges Each edge is represented by four elements pt is a pointer to information on the polygon (plane equation, shading, …)

8. AET Example

9. AET Example Notes Our edge table must contain data which enables us to look up z at at intersection point From x1 to x2 only A is considered. At x2 A and B are considered –plane equations are solved to get depth at (x2, i) –A is closest so x2 to x3 is filled as A At x3 A finishes so we draw B from x3 to x4 At x4 B and C are considered, B in front etc…

10. Method analysis Drawbacks –Sorting is required into the AET –If the number of polygons is high, the z-computation will be costly Accelerations Would usually store the scanning in a 1D BSP tree for large numbers of polygons! Exploit coherence (assume similar overlapping at y+1) Pre-order the polygons (no need to compute the depth calculations)

11. Zbuffer Don’t bother with per span tests - just test every pixel Most common usage is a full window sized array ZBUF (M*N) of 16, 24 or 32 bit “depth” values Basic idea: –Initialise Zbuffer to Z_MAX –For each polygon Point (x,y,z) of the polygon projects on pixel (x s,y s ) and has colour col associated If z < ZBUF[x,y] set CBUF[x,y] = col else do nothing

12. Use of the polygon scan-line renderer We can do this in several ways 1D z-buffer re-used on each scan line –Process each polygon with separate AET – Or use as adjunct to extended AET for multiple polygons Problems … –Aliasing on depth (z-buffer tearing)

13. Scanning Depth into the Zbuffer Now we have to write a z-value for each point –directly from plane equation (re-calculate for each point) –incremental across the scan-line (store z_start and dz) –Interpolate We see this one in more detail

14. Interpolating Depth Interpolate z along edges AND interpolate between edges on each scan-line (bi-linear interpolation) (X1,Y1,Z1) (X2,Y2,Z2) (XL,YL,ZL) (XR,YR,ZR)

15. Zbuffer Fill Example General form of ET –(y2,x1,dx/dy,z1,dz/dy) ET[1] = –ac (7,3,1/6,1,3/6) –ab (4,3,4/3,1,1/3) ET[4] = –cb (7,7,-3/3,2,2/3) a=(3,1,1) b=(7,4,2) c=(4,7,4)

16. …Contents of AET Scanline y=1 –ac (7,3,1/6,1,3/6) –ab (4,3,4/3,1,1/3) –zspans 1 to 1 y=2 –ac (7,3.166,1/6,1.5,3/6) –ab (4,4.333,4/3, 1.333, 1/3) –zspans 1.5 to 1.333 y=3 –ac (7,3.333,1/6,2.0,3/6) –ab (4,5.666, 4/3, 1.666, 1/3) –zspans 2 to 1.666

17. Recursive Subdivision Visibility Init: Clipping rectangle (CR) = window The CR is covered by a polygon that is in front of all others (draw it) The CR does not intersect a polygon (draw nothing) The CR overlaps a single polygon (draw it clipped) The CR contains a single polygon (draw it unclipped) ELSE quarter CR and repeat

18. Trade-Offs Zbuffer can be inaccurate with few bits –really simple to implement though! Scan-line AET good for large polygons –good coherency across lines –requires non-intersecting polygons zbuffer good for small, sparse polygons –AET more time consuming to maintain

19. Choosing the colour Gouraud shading Phong shading Texture mapping

20. Gouraud Shading Recall simple model for local diffuse reflection Gouraud interpolates this colour down edges and across scan-lines in the same manner as we just did for depth

21. Gouraud Details ET now contains –(y2,x1,dx,z1,dz,r1,dr,g1,dg,b1,db) (we are running out of registers!) Problems –not constant colour on rotation of points –misses specular highlights

22. Phong Shading Include specular component Interpolate normals across the scan-line instead of colours Recaptures highlights in the centre of polygons

23. Texture mapping We have seen that a colour can be assigned to a polygon, or to each vertex Now if we consider small details, we may not want to add polygons to represent every detail We prefer to keep a large polygon and use an image to represent the details

24. Example

25. Texture Mapping Why? –Approximation for surface colouring –Efficient packing of flat detail Standard texture mapping modifies diffuse mapping –Pasting a picture onto the polygon A texture is a 2D array of texels storing RGB or RGBA components

26. Difference between pixels and texels There can be a different match between the pixels of the display window and the texel of the texture

27. Overview Texture mapping –Inverse and Forward Mapping –Bilinear interpolation –Perspective correction Mipmapping Other forms of mapping –Environment –Bump mapping

28. Inverse Mapping Each vertex is associated with a point on an image (u,v)

29. Forward Mapping For points in the image, map onto the polygon –much harder to implement correctly, and harder to model Inverse mapping is much more commonly used –Most 3D modelers output U,V co-ordinates for texture application

30. Quick and Dirty Solution Yet more parameters in the ET! Bilinear interpolation of u&v down polygon edges, and across scan-lines Works, but is very ugly –Does not consider fore-shortening of image in depth

31. The Problem Same problem exists with Phong and Gouraud shading, but it MUCH less noticeable z correctz incorrect

32. The Solution Need to compensate for fore-shortening by interpolating over 1/z Interpolate (u’,v’,q) where q=1/z –At vertices we know (u 1,v 1,z 1 ), (u 2,v 2,z 2 ) –Interpolate between (u 1 / z 1,v 1 / z 1,1/z 1 ) and (u 2 /z 2,v 2 / z 2,1/z 2 ) to get (u’,v’,q) –Restore u,v, by dividing u’ and v’ by q

33. Sanity Check p0=(10,5,10,0,0) p i =(x i,y i,z i,u i,v i ) p1=(15,15,20,0,1) scanlines Y=5 Y=10 Y=15 At p0 (u’,v’,q) = (0,0,0.1) At p1 (u’,v’,q) = (0,0.05,0.05) On scanline 10 (u’,v’,q) = (0,0.025,0.075) (u,v) = (0,0.333) z = 1/0.075 = 13.33

34. Minor Issues Now have two divisions per pixel! Some optimisations –only do the division at end of the spans and interpolate across spans –or only do the division every n pixels Remaining problem –we have not touched upon how to clip u,v values in 3D or 2D!

35. Major Issues Picking your pixel! image

36. Sampling A pixel maps to a non-rectangular region Usually only perform map on centre of pixel Still have a problem of over-sampling (same texel maps to several pixels) or under-sampling the image (pixels only sparsely sample the texels)

37. Filtering Nearest neighbour Pick pixel with closest centre Bilinear Weighted average based on distance to pixel centre

38. Filtering Bilinear filtering solves (partially) the oversampling problem since it provides smooth shading between pixels scanline

39. Mip-Mapping When undersampling we use mippmapping Resample image at lower resolution Create a “pyramid” of textures. Interpolate texture between two adjacent layers

40. Texture Pyramid 128x128 64x64 32x32 1x1...

41. Sampling Choose two layers based on texel span –Choice is made selecting the two levels where the dv and dv for dx and dy are closest to one Interpolate between four pixels in higher layer and one in lower layer

42. Environment Mapping Don’t modulate diffuse colour modulate colour based on normal Effectively creates a reflective object The environment map is a surrounding texture

43. Bump Maps Use a map to perturb the normal for specular lighting