1. 1 by: Ilya Melamed Supervised by: Eyal Sarfati High Speed Digital Systems Lab

2. Background Project Objectives Description of the Algorithm Data Model Architecture design CAU Unit explanation CAU Unit statistics CAU Farm utilization Results Future Development Index

3. Raytracing calculates the light rays that are reflected, refracted, and obstructed by surfaces. Raytracing produces realistic and precise results, but it can be time–consuming. The technique is capable of producing a very high degree of visual realism, usually higher than that of typical Scanline rendering methods, but at a greater computational cost. Background vs

4. Main Objective: Implementing a real-time Ray-Tracing core on an FPGA card. Secondary Objectives:  Write an effective ray tracing algorithm  Test the algorithm in Matlab  Architecture design of parallel implementation for the algorithm.  HDL implementation of the architecture. Project Objectives

5. A scene is defined by objects, each object has: The object itself, made from polygons. Each object polygon is a triangle made of 3 vertices (3d vectors), normal vector, color, transparency and reflectivity values. A border, made from 12 polygons, avg. color, transparency and reflectivity. The border polygons don’t have color, transparency and reflectivity values. Also, the scene has information about point light sources, screen and camera location and resolution. Project Objectives Inputs

6. For each pixel Draw a ray from the eye (camera) through the pixel Check where the ray hits. Calculate shading Calculate reflective ray and passing ray Get color from reflected and passing ray by repeating the process for each of them Calculate color Display pixel color Description of the algorithm

7. 7 Make initial ray

8. 8 Calculate Shadowing

9. 9 Calculate reflective ray

10. 10 Calculate shadowing

11. 11 Calculate reflective and passing rays

12. 12 Calculate final color

13. Description of the algorithm

14. What is a “Shape Border”?

15. When looking at an object we will always see only the side of the object facing us, we will never see the back of someone facing us. The facing of a polygon can be checked before calculating the collision. This method requires that shapes are not flat, so that all polygons will have a facing. This will reduce the number of calculations by half for every scene. Description of the algorithm Why do shapes need to be full?

16. 16 Matlab Simulations Simulation of a single object with multiple colors, standing next to a mirror. Resolution: 250x250

17. 17 Matlab Simulations Simulation of a single object with multiple colors, standing alone. Resolution: 500x500, with internal shading.

18. 18 Matlab Simulations Simulation of three transparent objects each with a different color, good example of how light passes through objects and influences on other objects. Resolution: 500x500

19. 19 Data Model

20. 20 Data Model

21. 21 Data Model The fixed point data structure was chosen to make use of simpler arithmetic units. In order to prevent errors, the needed precision at the inputs is 0.125. In the full Ray Tracer algorithm errors are cumulative, and are inevitable for a large enough recursion stuck.

22. 22 Architectural Design DE3 Stratix III EP3SL150 (DE3-150) 142,000 logic elements (LEs) 5,499K total memory Kbits 384 18x18-bit multipliers blocks 736 user I/Os

23. 23 Architectural Design SOC Design NIOS II ON CHIP MEMORY DDR-II MEMORY VGA CTRL RAY TRACER AVALON

24. 24 Fetch RM Output Memory Ready bit Architectural Design *CAU – Collision Arithmetic unit *RM – Ray Manager + shading CAU Farm CAU Memory Parallel Protocol Serial Protocol Shading

25. 25 The CAU unit Interfaces:inputs: 576bit (72B) outputs: 417bit (52B) CAU 96 dot a dot b dot c normal origin point direction vector 32 96 success t collision point normal reflected ray passing ray clkstart finish started calculation reset CAU, collision arithmetic unit, a computational unit which calculates whether a ray hits a triangle polygon, and the other associated results.

26. 26 The result of a calculation can be either successful, i.e. there is a collision, or a failure, i.e. there isn’t a collision. The CAU is implemented as a multi-cycle machine with 3 multipliers and one divider. A successful calculation takes 26 cycles. Unsuccessful calculations take less cycles to complete and thus allow the average calculation time be less than the full 26 cycles. The CAU unit Implementation

27. 27 The CAU unit Implementation

28. 28 A single CAU unit uses: 28 18-bit DSP units out of 384 available on the Card, which means that the given DE3 card can have no more than 13 CAUs. 4796 combinational ALUTs, which are not a limiting factor. The CAU unit Utilization

29. 29 CAU Farm utilization RM CAU Farm CAU RM

30. 30 CAU Farm utilization

31. 31 Architectural Design Problems with the communication protocol The long time of the read process caused slow input rates from the Ray Managers to the CAU Farm, which caused low utilization of the CAU units. During Simulation, at most, 3 out of the available 8 CAUs were used, and most of the time, only two were used. Inserting the values in to the formula shows that 3 CAUs are enough to complete all the requests, as presented in the simulation.

32. 32 Results The CAU and the CAU Farm were implemented fully on HW (simulation, verification and synthesize). The Ray Manager was implemented partially on HW and fully on SW.

33. 33 Results CAU Farm CAU NIOS II ON CHIP MEMORY DDR-II MEMORY VGA CTRL AVALON

34. 34 Results The Final Result The final product is a hybrid HW and embedded SW. embedded C program running on the NIOS II processor. The C program implements a ray casting algorithm with shading, including all the adjustments. The C program implements a CAU function in software, which is used for verification purposes. The HW CAU is unit is faster than the SW emulation of the CAU, but this is a hindered by the limitations of the communication between the NIOS II processor and the hardware.

35. 35 Results The Final Result The final product can produce any scene of any resolution (with some small adjustments). The presented picture is a calculated 32x24 picture stretched out to a 640x480 picture. The calculation time for each frame on this example is ~0.5 minute using the SW CAU, and ~3.5 minutes using the HW CAU. The shading calculation is the only “meaningful” calculation made using software. All other calculations are either control calculation or collision calculations. All collision calculations are made using the CAU unit, including during the shading calculation. A CAU unit can calculate more than 4.4M calculations per second at full utilization.

36. 36 Results The Final Result A case study: Taking an example of a 76800 (320x240) pixels picture showing 1 shape with 100K polygons, 1 light source, recursion depth of 3 and hit ratio of 40% will give us ~7.5G collision calculations. If we are able to increase the working frequency of the CAU to 500 MHz, then to achieve a rate of 1 frame per second we will need 171 CAUs working at full efficiency. Increasing the frequency to 1 GHz will reduce the number to 86.

37. 37 Results The Final Result Starched 32x24 frames

38. 38 Results The Final Result 100x100 frames

39. 39 Future Development Increasing the work frequency of the CAU unit by using better technology. Reducing the number of cycles per calculations by using better calculation units (Divider). Incorporating the CAU unit into a GPU or a CPU using Ray Tracing only in part of a picture and the rest of the picture is made with Scan Line or other methods. Using the CAU unit to make other collision calculations for other algorithms.

42. 42 Architectural Design Communication Protocol In order to save on bus sizes the communication protocol between the Ray manager and the CAU Farm was chosen to be serial. The input bus is made of a 32bit and a 192bit (96x2) buses. The output bus is made of a 1bit, a 32bit and a 192bit buses. The read machine has 3 read states (cycles). The write machine has 2 write states (cycles).

43. 43 The CAU unit Logic Step 1: check if the polygon is facing the ray.

44. 44 The CAU unit Logic Step 2: find collision point with the plane of the polygon, and check that it is in front of the origin point (t>0).

45. 45 The CAU unit Logic

46. 46 Step 3: Check that the collision point is inside the polygon. For each pair of adjacent polygon vertices, check that the third vertex and the collision point are on the same half plane The CAU unit Logic

47. 47 Step 4: If all previous stages are successful, then we calculate the reflected ray. The CAU unit Logic source: http://graphics.stanford.edu/courses/cs148-10-summer/docs/2006--degreve--reflection_refraction.pdf

48. 48 The average number of cycles per calculation is: We get: The CAU unit Statistics

49. 49 The CAU unit Statistics

50. 50 Shading gives the amount of light which is reflected from the surface to the viewer. At first a light ray is “created” from the light source to the collision point. If the light ray is not abstracted, then the amount of light that reaches the eye is a function of the angle between the polygon and the direction vector: Shading