TI Information – Selective Disclosure IEEE Santa Clara ComSoc/CAS Weekend Workshop – Event-based analog sensing Theodore Yu Texas Instruments – Kilby Labs, Silicon Valley Labs September 29,

TI Information – Selective Disclosure Living in an analog world The world is analog –Many different levels to sense Sight, sound, touch, taste, smell –Analog interfaces are uniquely suited for each environment Increasingly, we turn to machines to help interpret the world for us –Interface through sensors and actuators with computation being performed in digital machines e.g. microprocessors, cellphones, CPUs, etc. –Digital computation is robust, easily configurable, and widespread 2

TI Information – Selective Disclosure Analog-digital interface 3 The placement of the boundary between analog and digital is flexible –But transitions are expensive All-digital approach: send raw sensor data to digital domain –Places the burden upon the analog-digital interconnect and digital processing power consumption All-analog approach: all-analog signal processing –Often highly task specific which increases development time and reduces generalization to other applications AD A D -Mostly digital Analog world is directly sampled into the digital domain –e.g. all-digital implementations -Mostly analog Analog world is processed and interpreted in analog –e.g. traditional analog implementations

TI Information – Selective Disclosure Analog-digital interface – smart sensors The placement of the boundary between analog and digital is flexible –But transitions are expensive –Smart sensors and actuators Learning and interpretation of analog information Adaptation in analog sensor and actuator operation 4 AD A D -Mostly digital Analog world is directly sampled into the digital domain –e.g. all-digital implementations -Mostly analog Analog world is processed and interpreted in analog –e.g. traditional analog implementations

TI Information – Selective Disclosure Analog-digital interface Since the transition from analog domain to digital domain is expensive, only transmit what is necessary. –Maximize information content of each digital bit –Minimize transfer of redundant information Analog sensor interface –Objective Operate analog circuits in high efficiency regime for low-power performance Integrated local analog signal processing circuitry results in sparse data being transferred to the digital domain –Extract features of interest from sensors in the analog domain –Transmit as digital events to the digital domain meaning? Analog to digital encoding

TI Information – Selective Disclosure Event-based sensing approach Each digital event encodes a feature of interest from the sensor –Event encoding Feature selection –Select what is and is not a feature from sensor data –Decide what feature information to transmit for each event (i.e. spatial position, temporal position, etc.) –Event decoding Digital processor must now interpret and understand what each event means Describes features of object as time- based digital events Analog to digital encoding

TI Information – Selective Disclosure Dynamic vision sensor (DVS) Frame-free image (scene) processing –Only transmits individual pixel information when has a change in relative log intensity Characteristics –Low bandwidth –Low power consumption –Low computational requirements –High sensor dynamic range Technical specifications –128x128 resolution, 120dB dynamic range, 23mW power consumption, 2.1% contrast threshold mismatch, 15us latency Lichtsteiner, et. al. (ISSCC 2006, JSSC 2008)

TI Information – Selective Disclosure A silicon retina that reproduces signals in the optic nerve 8 Zaghloul, et. al. (J. Neural Eng. 2006) Frame-free image (scene) processing –Only transmits individual pixel information when has a change in relative log intensity Event decoding scheme –ON activity corresponds to bright pixels and OFF activity corresponds to dark pixels Technical specifications –<100mW power consumption, 3.5mm x 3.3 mm

TI Information – Selective Disclosure Convolution chips for image processing Event-based image processing –Frame-free event-based image processing of asynchronous events –On-the-fly processing of events results in 2-D filtered version of the input flow Characteristics –Arbitrary kernel size and shape Technical specifications –32x32 pixel 2-D convolution event processor, 155ns event latency between output and input, 20Meps input rate, 45 Meps output rate, 350nm CMOS, 4.3x5.4mm 2, 200mW at maximum kernel size and maximum input event rate Linares-Barranco, et. al. (TCAS 2011)

TI Information – Selective Disclosure Silicon cochlea architecture Chan, et. al. (TCAS I 2007) Seek to emulate cochlea performance and functionality by emulating cochlea biological architecture in silicon -2 nd order LPF bank -Transform into analog signal -Transform into “digital” neural event signal Input sound Digital events -Each “event” is a data packet describing event source (LPF) and event time

TI Information – Selective Disclosure Reconstructed silicon cochlea data time channel number Silicon cochlea PC Input sound Digital events PC reconstructs the output digital event information by sorting by channel (LPF) number and then aligning according to time stamp information.

TI Information – Selective Disclosure 750 Hz pure tone Example data with pure tones (for one channel) 300 Hz pure tone Simple real-time data processing procedure Count the time difference between events (interspike interval, ISI) for each channel Arrange the ISIs into a histogram A peak in the ISI histogram indicates a resonant frequency response channel number time bin count ISI

TI Information – Selective Disclosure Sound Discrimination Example “coo” sound“hiss” sound Wav file FFT ISI histogram

TI Information – Selective Disclosure 3-D integrated silicon neuromorphic processor 14 65,000, two-compartment neurons –Conductance-based integrate and fire array transceiver ( IFAT ) 65 million, 32-bit “virtual” synapses –Conductance-based dynamical synapses –Dynamic table-look in embedded memory (2Gb DRAM ) Locally dense, globally sparse synaptic interconnectivity –Hierarchical address-event routing ( HiAER ) –Dynamically reconfigurable –Asynchronous spike event I/O interface Sender Receiver 5 mm DRAM HiAER (Digital CMOS) IFAT (Analog CMOS) Top metal TSV Top metal I/O pad HiAERIFAT 0.13  m CMOS Hierarchical address-event routing (HiAER) Park, et. al. (ISCAS 2012)

TI Information – Selective Disclosure Theodore YuUCSD Integrated Systems Neuroengineering Lab Event-driven framework Coincidence detection performs efficient spike- based computation –coincidence detection two or more arriving events result in a stronger response than a single arriving event –applications event-driven sensing –sensors are only “on” when something important happens event-driven computation –information is sparsely represented with events Yu, et. al. (EMBC 2012) Provide background on motivation Event-based approach relies upon temporal encoding to communicate signals. The time of the event is the key parameter, not the voltage value. Event-encoding is robust against additive noise.

TI Information – Selective Disclosure Temporal code and synchrony At a local scale, neurons perform coincidence detection within temporal integration window. At a network scale, the temporal delay information in events models the spatial distribution between neurons. –Each scene of interest can be encoded as a unique combination of features 16 10ms delay 5ms delay 4ms delay Coincidence? Yes or no? Input pattern

TI Information – Selective Disclosure Temporal code and synchrony example 17 10ms delay 5ms delay 4ms delay Coincidence? Yes! 10ms delay 5ms delay 4ms delay Coincidence? No! Event at t = 3ms Event at t = 8ms Event at t = 7ms Event at t = 2ms Event at t = 8ms Event at t = 7ms

TI Information – Selective Disclosure Summary Analog event-based sensing –Since the transition from analog domain to digital domain is expensive, only transmit what is necessary. Maximize information content of each digital event through encoding of features in analog domain Minimize transfer of redundant information for sparse digital signal processing –Applications Visual and acoustic sensors for event-encoding of features Event-based processor performs event-decoding of features utilizing coincidence detection in neural synchrony 18

TI Information – Selective Disclosure 19 Thank you