1. SIMD Image Processor Eric Liskay Andrew Northy Neraj Kumar 1

2. Features SIMD architecture Large array of cell processors operating in parallel Each processor performs the same function on a kernel -Kernel size is a 3x3 array of pixels Number of cell processors equal to (Tile Size in kernels) 2 -For example, if the tile size is 10 kernels, each tile will be 30x30 pixels Convolution Mode 3x3 pixel matrix read into the cell, only center pixel is written out Used for minimum, maximum, average, and edge detection operations User Inputs User can specify parameters for different functions User can specify a custom kernel for convolution 2

3. Top Level Diagram A B Cell Processo r x10 C Pixel Buffers A & B Pixel Buffer C (Output) 900x24-bits A - Pixel Data B - Pixel Data 900x24-bits Opcode XY Pixels User Inputs 24-bit Color RAW Image 3

4. Data Structures Pixel 3x3 Kernel Tile Red Channel Green Channel Blue Channel 8 bits 4

5. Image Processor Controller 4 Processes Load images from files into pixel buffers Positive Edge of Clock: Send data to cell processors Negative Edge of Clock: Read image from cell processors into Pixel Buffer C Write Output to file when finished processing the image

6. Cell Processor Each Cell processor operates on 3x3 pixel array, each pixel has 3 color channels -Two image inputs -Two user inputs -Opcode input -Kernel input (user’s filter) -One image output 6

7. Cell Functions Arithmetic and logical operations -Add, subtract, multiply divide -AND, OR, NOR, XOR -Operands may be two images, or one image and user input -Operation can be limited to one color channel or all channels 7

8. Cell Functions Darken Highlights -Compare sum of color channels with threshold value -Decrease value from all channels by set amount if sum is larger than threshold Brighten Shadows -Compare sum of color channels with threshold value -Increases value from all channels by set amount if sum is smaller than threshold 8

9. Cell Functions Convolution -Sobel, Prewitt, and Robinson Edge Detection -User specified Kernel -3x3 Minimum, Maximum, and Averaging Filters 9

10. Edge Detection Using the Sobel Operator The Sobel operator uses two 3x3 kernels which are convolved with a 3x3 array of pixels from the image A is the 3x3 array of pixels from the image G x and G y are the derivatives of the convolutions is the convolution operator G is the gradient magnitude 10

11. The Opcodes Regular Processing Mode (9 pixels in, 9 pixels out) MneumonicOpcodeDescription add0x00add pixel channel values in array A to array B addi0x01add userInputA to values in all channels from array A addir0x02add userInputA to value in red channel from array A addig0x03add userInputA to value in green channel from array A addib0x04add userInputA to value in blue channel from array A sub0x05subtract pixel channel values in array B from array A subi0x06subtract userInputA from values in all channels from array A subir0x07subtract userInputA from value in red channel from array A subig0x08subtract userInputA from value in green channel from array A subib0x09subtract userInputA from value in blue channel from array A mult0x0Amultiply pixel channel values in array A by array B mult20x0Bmultiply pixel channel values in array A by 2 multi0x0Cmultiply pixel channel values in array A by shifting left by userInputA div20x0Ddivide pixel channel values in array A by 2 divi0x0Edivide pixel channel values in array A by shifting right by userInputA inv0x0FInvert pixel channel values in array A and0x10AND pixel channel values in Array A with pixels in array B or0x11OR pixel channel values in Array A with pixels in array B nor0x12NOR pixel channel values in Array A with pixels in array B xor0x13XOR pixel channel values in Array A with pixels in array B drkn0x14Darken Highlights: If sum of pixel channel values in array A are above userInputA, subtract userInputB brtn0x15Brighten Shadows: If sum of pixel channel values in array A are below userInputA, add userInputB grayr0x16Grayscale based on Red Channel grayg0x17Grayscale based on Green Channel grayb0x18Grayscale based on Blue Channel pixflip0x19Pixels mirrored over middle pixel 11

12. The Opcodes Convolution Mode (9 pixels in, 1 pixels out) MneumonicOpcodeDescription sobel0x1ASobel sobel20x1BSobel Method 2 prewitt0x1CPrewitt robn0x1DRobinson North robnw0x1ERobinson Northwest robne0x1FRobinson Northeast robe0x20Robinson East robw0x21Robinson West robs0x22Robinson South robse0x23Robinson Southeast robsw0x24Robinson Southwest avggry0x253x3 pixel average in grayscale avgclr0x263x3 pixel average in color ukernel0x27User programmable filter min0x283x3 pixel minimum max0x293x3 pixel maximum 12

13. Color Codes

14. Regular Processing Mode 14

15. Red Image outputC(x,y)(0) <= B"11111111”; outputC(x,y)(1) <= inputA(x,y)(1); outputC(x,y)(2) <= inputA(x,y)(2); OriginalProcessed 15

16. Red Image outputC(x,y)(0) <= inputA(x,y)(0); outputC(x,y)(1) <= B"11111111"; outputC(x,y)(2) <= inputA(x,y)(2); OriginalProcessed 16

17. Red Image outputC(x,y)(0) <= inputA(x,y)(0); outputC(x,y)(1) <= inputA(x,y)(1); outputC(x,y)(2) <= B"11111111"; OriginalProcessed 17

18. Red = 0 OriginalProcessed 18

19. Green = 0 OriginalProcessed 19

20. Blue = 0 OriginalProcessed 20

21. Add: Image A + Image B OriginalProcessed 21

22. Add Immediate: Image A + User Input A OriginalProcessed 22

23. Add Immediate Red: Image A(red channel) + User Input A OriginalProcessed 23

24. Add Immediate Green: Image A(green channel) + User Input A OriginalProcessed 24

25. Add Immediate Blue: Image A(blue channel) + User Input A OriginalProcessed 25

26. Subtract: Image A - Image B OriginalProcessed 26

27. Subtract Immediate: Image A - User Input A OriginalProcessed 27

28. Subtract Immediate Red: Image A(red channel) - User Input A OriginalProcessed 28

29. Subtract Immediate Green: Image A(green channel) - User Input A OriginalProcessed 29

30. Subtract Immediate Blue: Image A(blue channel) - User Input A OriginalProcessed 30

31. OriginalProcessed Multiply: Image A * Image B 31

32. Multiply by 2: Image A * 2 OriginalProcessed 32

33. Multiply Immediate: Image A * User Input A OriginalProcessed 33

34. Divide by 2: Image A / 2 OriginalProcessed 34

35. Divide Immediate: Image A >> User Input A OriginalProcessed 35

36. Invert: Image A XOR “11111111” OriginalProcessed 36

37. AND: Image A AND Image B OriginalProcessed 37

38. OR: Image A OR Image B OriginalProcessed 38

39. NOR: Image A NOR Image B OriginalProcessed 39

40. XOR: Image A XOR Image B OriginalProcessed 40

41. Darken Highlights: Decreases brightness by (User Input B) if average value of the pixels is above User Input A OriginalProcessed 41

42. Brighten Shadows: Increases brightness by (User Input B) if average value of the pixels is below User Input A OriginalProcessed 42

43. Convert to Grayscale using Red Channel OriginalProcessed 43

44. Convert to Grayscale using Green Channel OriginalProcessed 44

45. Convert to Grayscale using Blue Channel OriginalProcessed 45

46. Pixel Flip OriginalProcessed 46

47. Convolution Mode 47

48. Sobel 48

49. Sobel Method 2 49

50. Prewitt 50

51. Robinson North 51

52. Robinson Northwest 52

53. Robinson Northeast 53

54. Robinson East 54

55. Robinson West 55

56. Robinson South 56

57. Robinson Southeast 57

58. Robinson Southwest 58

59. 3x3 Pixel Average Grayscale 59

60. 3x3 Pixel Average Color 60

61. User Programmable Filter 61

62. 3x3 Pixel Minimum OriginalProcessed 62

63. 3x3 Pixel Maximum OriginalProcessed 63

64. Make Image Processor synthesizable Move loading files into testbench(HVL) and separate from image processor and Cells(HDL) Veloce? Enable the processing of larger images by loading and writing only part of the image at a time Other convolution ideas(e.g. Noise Reduction) Parameterized Kernel Size to enable larger kernels Gradient Threshold on “Darken Highlights” and “Brighten Shadows” for a less abrupt transition Multiple Operations done on image before writing it Future Work 64

65. Horizontal Flip OriginalProcessed 65

66. Vertical Flip OriginalProcessed 66

67. Rotate 180° Clockwise OriginalProcessed 67

68. All Photographs © Eric Liskay Lecture 10. Edge detection. Sobel and similar filters. Convolution http://web.cecs.pdx.edu/~mperkows/CLASS_VHDL_99/2012/2012-lecture010_edge-detection-SobelG.ppt Wikipedia - Sobel Operator http://en.wikipedia.org/wiki/Sobel_operator References 68