Eye-RIS
Vision System sense – process - control autonomous mode Program stora
Outline hardware system overview AnaFocus, SIS Q-Eye chip DiCop ALTERA NIOS II RISC uP, MLP software development environment
System overview Early-processing Extract information from the image flow Output: reduced set of data Pixel-wise simultaneous processing Not a loop! Post-processing Inputs are abstract entities Output complex decisions
System overview Two main parts: Smart Image Sensor (SIS) – Q-EYE chip RISC processor – Altera Nios
Q-Eye chip 176 x 144 (QCIF) resolution Each cell: Multi-mode optical sensors Pixel memories Linear and non-linear analogue processors Binary processors Interconnected with 8 neighbours
Q-Eye chip „This means that each pixel can both sense the corresponding spatial sample of the image and process this data in close interaction and cooperation with other pixels.” „Smart Image Sensor Q-Eye is also called Focal-Plane Processor (FPP) because it processes images at the same physical layer where they are sensed.”
Conventional Vision System Be aware of the data size of F!
Eye-RIS Vision System In this case: f << F
Architecture of the Q-Eye chip
The cell array 176x144 interconnected sensing-processing cells. Each of these cells contains mixed-signal circuitry capable of: Sensing images; including adaptive linear and high-dynamic range acquisition Storing up to 7 grey scale images and up to 4 binary images Performing grey-scale image additions, subtraction and multiplication by a constant Performing spatio-temporal image filtering (low-pass, high pass, band pass and stop band) with programmable bandwidth. Performing grey scale image convolutions Performing grey scale to binary, and binary to grey scale conversions Performing generic combinational and sequential binary operations Performing Hit-and-Miss binary operations Real-time loading and downloading of images Ultra fast calculation and input/output of certain image features such as centroids, mean values, pixel coordinates…
Block Diagram of a Q-Eye cell
Altera Nios II General-purpose RISC processor core. Harvard architecture. Full 32-bit instruction set, data path, and address space. 32 general-purpose registers. 32 external interrupt sources. Single-instruction 32 x 32 multiply and divide producing a 32-bit result. Single-instruction barrel shifter. Access to a variety of on-chip peripherals, and interfaces to off-chip memories and peripherals through Avalon Bus. Hardware-assisted JTAG debug module enabling processor start, stop, step and trace. Instruction and Data Cache Memory. JTAG Debug Module. Exception and interrupt controller.
Summary of Q-Eye chip Chip: Q-Eye Technology: 0,18 um Power Supply: Dual (1.8 V, 3.3 V) # of Processor Elements array: 176×144 Accuracy: ~8bit Density of Processor Elements: 422 PE × mm 2 Die Size: 60 mm 2 Package size: 64×67×63 mm Power Consumption of system: 700mW Maximum frame rate: > I/O type: Binary or grayscale type images
Software Eye-RIS ADK Nios II: ANSI C/C++ Q-EYE: CFPP
Programming
Programming Q-Eye chip The CFPP Image Processing Library (IPL) Spatio-temporal filters Arithmetic operation between images Thresholding Morphological and logic binary operations The Extended Image Processing Library (EIPL) Blob management Classifying functions Linear and non-linear digital processing of grey-level images Geometrical transformations
Programming Altera Nios II C,C++, Assembly The Eye-RIS Basic Library (EBL): Control the execution of the CFPP code I/O management Sending and/or receiving images to/from the PC either to be displayed or saved to disk Printing error or information messages to a console Timer management
Programming Eye-RIS (sync)
Programming Eye-RIS (async)
CFPP global memories // Declaration extern fpp_int value; extern fpp_bool flag; extern fpp_time texp; // Access void foo() { value = 123; int foo = value; if(flag) { FPPTime_write(texp, 500); int readTExp = FPPTime_read(texp); }
DEMO