1 Brain Neuroinformatics: Auditory System February 27, 2002 Soo-Young Lee Brain Science Research Center Korea Advanced Institute of Science & Technology bsrc.kaist.ac.kr

KAIST Brain Science Research Center 2 Artificial Auditory System Based on Human Cognitive Mechanism  Develop mathematical model and auditory chip  Develop Continuous speech recognition system  Auditory model  Binaural processing  Selective attention  Continuous speech recognition 95% recognition in 10dB SNR

KAIST Brain Science Research Center 3 Auditory Pathway Superior Olive Cochlea Superior Olive Cochlea Higher-Order Brain Function (Recognition) Cochlear Nucleus Cochlear Nucleus Auditory Cortex Auditory Cortex   Improvement of Cochlea model  Binaural hearing model  Auditory cortex model  Selective attention model  Continuous-speech recognition system  Speech-recognition chip

KAIST Brain Science Research Center 4 Object Path Attention Spatial Path Higher- Order Functions Top-Down Attention Bottom-Up Attention TD/BU Attention Cross Correlation Sound Localization Speech Enhancement Speaker Separation Nonlinear Features Time Adaptation Complex Sound Time-Freq. Masking Speech Rec. System Parser Cochlea Aud. Cortex Research Scopes Auditory Pathway

KAIST Brain Science Research Center 5 Masking - Lateral Inhibition - Recursive

KAIST Brain Science Research Center 6 Masking using Lateral Inhibition

KAIST Brain Science Research Center 7 Temporal Masking

KAIST Brain Science Research Center 8 Temporal Masking: Integration Model

KAIST Brain Science Research Center 9 Time & Frequency Response

KAIST Brain Science Research Center 10 Isolated Word Recognition Rates

KAIST Brain Science Research Center 11 Discussions: Masking Masking Suppresses Unwanted Noisy Components in Signal Simultaneous Masking by Lateral Inhibition - Recognition performance was enhanced with MFCC model - Proposed model can be used with any auditory model - ZCPA has spectral masking effect Temporal Masking by Unilateral Inhibition - Unilateral inhibition using the integration model - Model resembles other feature processing algorithms - Recognition performance was enhanced with RASTA parameters

KAIST Brain Science Research Center 12 ASR in mismatched environments Environmental information Background noise, acoustic/transmission channel Assume environment degradation model Channel Compensation Clean speech Channel Additive noise Distorted speech

KAIST Brain Science Research Center 13 Mapper train Where and which type of mapper should be deployed? Speaker-to-Microphone mapping F.E. + clean F.E.Mapper distorted Error F.E.Trained Mapper distorted To recognizer

KAIST Brain Science Research Center 14 Adaptive Noise Cancelling Adaptive noise cancelling An approach to reduce noise based on reference noise signals System output The LMS algorithm

KAIST Brain Science Research Center 15 ICA-based Approach to ANC The difference between the LMS algorithm and the ICA-based approach Existence of the score function The LMS algorithm Decorrelate output signal from the reference input The ICA-based approach Make output signal independent of the reference input Independence Involve higher-order statistics including correlation The ICA-based approach Remove the noise components using higher-order statistics and correlation

KAIST Brain Science Research Center 16 TDAF approach to ANC Normalized LMS algorithm Normalized ICA-based algorithm where

KAIST Brain Science Research Center 17 Experimental Results (1) Experiments for artificially generated i.i.d. signals SNRs of output signals for the simple simulation mixing filter (dB) Signal and Noise Initial SNRs SNRs after convergence LMS algorithm ICA-based approach Laplacian Gaussian

KAIST Brain Science Research Center 18 Experimental Results (2) Experiments for recorded signals Signal waveforms for the car noise and the simple simulation filter Signal sourceNoise source Primary input signalSystem output signal

KAIST Brain Science Research Center 19 Experimental Results (3) Experiments for recorded signals SNRs of output signals for the measured filter

KAIST Brain Science Research Center 20 Experimental Results (4) Comparison of learning curves with and without TDAF Car noise The ICA-based approachThe LMS algorithm

KAIST Brain Science Research Center 21 Discussion: ANC A method to ANC based on ICA was proposed. The ICA-based learning rule was derived. The ICA-based approach Include higher-order statistics Make the output independent of the reference input The LMS algorithm Make the output uncorrelated to the reference input Gave better performances than the LMS algorithm TDAF method was applied to the ICA-based approach. Derived the normalized ICA-based learning rule Improved convergence rates

KAIST Brain Science Research Center 22 Bottom-Up and Top-Down Attention Attended Output Top-Down Expectation Bottom-Up Recognition Environment External Cue Brain Classifier Output Internal Cue Input Features Attended Input Input Stimulus Bottom-Up Attention  Bottom-Up: - Masking - ICA  Top-Down: - MLP - HMM

KAIST Brain Science Research Center 23 디지털 음성 인식 프로세서 (1) 실시간 음성 인식 프로세서 인텔 8051 프로세서 채택 ( 범용성 ) 12 MHz 동작 : 1.93 MIPS 하드웨어 특징 추출 (16 채널 ) 고속 인식을 위한 인식 가속기 50 단어 시 20 ms 이내 인식 결과 출력 인식률 95% 이상

KAIST Brain Science Research Center 24 디지털 음성 인식 프로세서 (2) Hynix 0.35 공정 패키지 형태 : 64-TQFP 내부 메모리 : 128x12, 256x8, 2048x8 AGC, A/D, D/A 내장 12KHz, 12 bit A/D 12Khz, 8 bit D/A 인식을 위한 특수 레지스터 특징 추출 레지스터 인식 가속 레지스터 인식을 위한 효율적인 구조 AGCA/DD/A 2048x8 128X12 256X8 CM8051

KAIST Brain Science Research Center 25 디지털 음성 인식 프로세서 (3) CM8051 블록도 및 I/O 구성도

KAIST Brain Science Research Center 26 디지털 음성 인식 프로세서 (4) 기본 구성 회로도

KAIST Brain Science Research Center 27 Analogue ICA Chip   multiplier   current summation r  learning rate 4 x 4 ICA network in one Chip

KAIST Brain Science Research Center 28 Test Results in Waveforms Source Signals (s1, s2) two different male’s voice 16 kHz sampled Mixed Signals (x1, x2) Instantaneous mixture Mixing Mtx A is Separated Signals (o1, o2) Recovered original sources

KAIST Brain Science Research Center 29 Fabricated Chip 2.8mm x 2.8mm AMS CMOS 0.6um 2 poly-3 metal Analog Digital Hybrid Process Die Photo of a Fabricated ICA Chip