1. Lecture 7 The Eye and Neuromorphic Vision 1

2. Outline The eye and the retina Artificial Vs biological vision systems Fundamentals of photo receptors Read-out strategies Examples of neuromorphic vision systems Further processing 2

3. Eyeball Cross Section and Retina 3

4. Retina Cells 4

5. Schematized Retinal Cells 5

6. Outline The eye and the retina Artificial Vs biological vision systems Fundamentals of photo receptors Read-out strategies Examples of neuromorphic vision systems Further processing 6

7. Frame-based Sensors Vs Biological Vision Artificial VisionBiological Systems Frame-basedContinuous Detection Global AGCLocal AGC RedundancyLow redundancy High power consumptionLow power consumption 7

8. Notion of Frame T frame T integration Time T integration <T frame 8

9. Dynamic Range Rüedi et al, JSSC 2003 A 128x128 Pixel 120dB Dynamic-Range Vision-Sensor Chip for Image Contrast and Orientation Extraction Bad choice of the integration time The human eye has a dynamic range higher than 10 decades 9

10. Outline The eye and the retina Artificial Vs biological vision systems Fundamentals of photo receptors Read-out strategies Examples of neuromorphic vision systems Further processing 10

11. Depletion Region and Phototransduction 11

12. Silicon Electromagnetic Spectrum 12

13. Some Possible Photodetectors in CMOS Technology 13 A. Moini. Vision Chips. Kluwer Academic Publishers, 2000.

14. Photogates 14

15. Logarithmic Photoreceptors (a) (b) 15

16. Photoreceptors with Negative Feedback (a) (b) 16

17. Classic Active Pixel Sensor (APS) 17

18. Outline The eye and the retina Artificial vision Vs biological systems Fundamentals of photo receptors Read-out strategies Examples of neuromorphic vision systems Further processing 18

19. Read-out Strategies: Addressing/Scanning Charge Coupled Devices (CCD) Address Event Representation (AER) 19

20. Addressing/Scanning Token System 20

21. Charge Coupled Devices (CCD) 21

22. Address Event Representation (AER) (I) 22

23. Address Event Representation (AER) (II) 23

24. Outline The eye and the retina Artificial Vs vision biological systems Fundamentals of photo receptors Read-out strategies Examples of neuromorphic vision systems Further processing 24

25. Delbrücks Adaptative Photo Cell (I) 25

26. Delbrücks Adaptative Photo Cell (II) Non-linear element 26

27. Mahowalds Pixel 27

28. Retina Pixel Boahen 28

29. Dynamic Vision Sensor (DVS) P. Lichtsteiner, C. Posch, and T. Delbrück, A 128 128 120 dB 15 µs latency asynchronous temporal contrast vision sensor, IEEE J. Solid-State Circuits, vol. 43, no. 2, pp. 566–576, Feb. 2008. 29

30. Common Source Amplifier 30 Can used as comparator

31. Examples of Biomorphic Vision Sensors Spatio-Temporal Contrast Detector: http://folk.uio.no/juanle/Projects/Spatial_contrast_retina.html Dynamic Vision Sensor: http://folk.uio.no/juanle/Projects/DVS.html 31

32. Outline The eye and the retina Artificial vision Vs biological systems Fundamentals of photo receptors Read-out strategies Examples of neuromorphic vision systems Further processing 32

33. Further Image Processing Motion Detection Features extraction 33

34. Token Based Motion Processing: Reichardt Detector 34

35. Intensity Based 35

36. Features Extraction 36

37. Difference of Gaussians Kernel (I) 37

38. Difference of Gaussians Kernel (II) 38

39. Result of Edge Detection 39

40. 45 Degrees Edge Kernel (I) 40

41. 45 Degrees Edge Kernel (II) 41