MTA-SZTAKI Roboteyepair an ultra-high speed stereo vision system based on CNN technology

MTA-SZTAKI Motivation The mechanical properties of robots enable faster operation than their current vision systems; A high-speed, medium resolution, low cost vision system has many potential application fields in the industry; Both stereo and mono versions have potential applications;

MTA-SZTAKI Goals Main characteristics of the Roboteyepair: –Standalone vision system; –Capable to capture and process image pairs at frame rates up to 5000 per second; –Medium resolution; –High performance focal plane CNN processors; –Equipped with serial and Ethernet communication ports; –Developed for industrial use.

MTA-SZTAKI The CNN visual processor 128x128 sensor-processor array; Optimized for performing array operations on analog and logic values; Time to perform a typical operation: –Logic: 300ns (morphology, binary logic operations, skeletonizing, etc.) –Analog: 3-5ms (diffusion, edge detection, Sobel, etc..) 8 internal image memories; 128Mpixel/sec I/O speed (128 internal AD/DA) 0.35mm technology, 12x12mm silicon surface; Operates as a coprocessor, a master processor is needed

MTA-SZTAKI Block diagram of the system

MTA-SZTAKI Realization of the system

MTA-SZTAKI The finished boards

MTA-SZTAKI The complete system

MTA-SZTAKI Programming the DSP Image processing –Native languages (AMC, Alpha, C) Handling two CNN chips –The target chip has to be specified for every command –Synchronization Adapting CNN image processing library Communication using the ETRAX chip

MTA-SZTAKI Programming the communication processor ETRAX LINUX kernel –Driver (ETRAX  XBUS) TCP/IP communication module (INET Daemon) –Assigns the communication driver to the given TCP port –standard input  standard output redirecting A special daemon has also been implemented –Boot device

MTA-SZTAKI Monitoring software on the PC side Why is it needed: –Supporting system setup –Application development and testing –Parameter adjustment –Testing camera parameters Capabilities: –Multi-window graphical interface –Editor, compiler and software downloading devices

MTA-SZTAKI The project will be finished in May, 2003

MTA-SZTAKI Stereo machine vision system for robots; Visual navigation for NASA Mars Mission 2007 Non-invasive speed and displacement measurement, quality control of narrow textile bands and tapes (textile industry); Classification of large number of tiny objects (pills, grains, pebbles, etc.); Monitoring fast industrial processes (sparkplug); Applications

MTA-SZTAKI Replace the current DSP with a TMSC DSP MHz Equip the system with an optional 1.3 MPixel CMOS sensor Support of communication via USB port These developments are expected to be finished by September, 2003 Future developments

MTA-SZTAKI Thank you for your attention!