Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Keyword Spotting Using Crosscorrelation Presenters: Bathiya Senevirathna Roshan Rajeev Roshan RajeevAdvisor: Dr. Uchechukwu Abanulo Montgomery College Speech Processing Laboratory

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Applications of Research Method Results Demo Presentation Outline Presenters: Bathiya Senevirathna Roshan Rajeev Roshan RajeevAdvisor: Dr. Uchechukwu Abanulo Montgomery College Speech Processing Laboratory

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Keyword detection Did the speaker say ____? Keyword location Where did the speaker say ____? Research Goal Applications of Research Method Experiment Demo

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Applications of Research Research Goal Applications of Research Method Experiment Demo

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Interactive Voice Response Telephone ticket booking National Security Conversation monitoring to identify words of interest Research Goal Applications of Research Method Experiment Demo

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Method Research Goal Applications of Research Method Experiment Demo

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Crosscorrelation Measure of similarity between two signals Two signals compared by Sliding one signal by a certain time lag Multiplying both the overlapping regions Repeating the process and adding the products until there is no more overlap If both signals are exactly the same, there’s a maximum peak at the time = 0, and the rest of the correlation signals tapers off to zero Research Goal Applications of Research Method Experiment Demo Area Overlap

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Applications of Research Method Experiment Demo Crosscorrelation

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Applications of Research Method Experiment Demo Typical Cross-Correlation Results Keyword Match No Match See any differences??

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Applications of Research Method Experiment Demo A Closer look… Much higher amplitude near zero-lag pointMuch higher amplitude near zero-lag point Rest of graph is almost zeroRest of graph is almost zero No clear maximum pointsNo clear maximum points Amplitude is generally the same throughoutAmplitude is generally the same throughout Keyword Match No Match

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Applications of Research Method Experiment Demo The Algorithm 1. 1.Find average of points in the first 10% of the samples 2. 2.Find average of points in the last 90% of the samples 3. 3.Compute the ratio of the two values. If n = the number of samples in the crosscorrelation graph: Ratio =

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Applications of Research Method Experiment Demo The Algorithm xcorr 3. Shift observed portion by a small amount and repeat process If a portion is reached where the calculated ratio is above a defined minimum threshold then mark the location of the indices 2. If a portion is reached where the calculated ratio is above a defined minimum threshold then mark the location of the indices 1. Let the length of the keyword or phrase be n. The cross correlation of the keyword and the first n samples of the utterance is computed.

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 The Experiment Research Goal Applications of Research Method Experiment Demo

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Applications of Research Method Experiment Demo Experiment Effectiveness of the algorithm in finding keywords in a speech utterance 8 speakers, mixed gender Threshold varied from 6 to 15 Criteria: Hit: >50% of keyword length found in correct location False Alarm: 2 x length of keyword found in wrong location

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Applications of Research Method Experiment Demo Criteria - Hits Actual Location Wrong Location Miss! <50% of Word Found Miss! >50% of Word Found Hit!

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Applications of Research Method Experiment Demo Criteria – False Alarms Actual Location (No Keyword) >2x Keyword Length Found False Alarm! <2x Keyword Length Found No False Alarm! No Keyword Found No False Alarm!

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Applications of Research Method Experiment Demo Results Performance of Algorithm at Different Threshold Levels At a threshold of 8: 4/8 keywords were found 1/8 utterances with false alarms At a threshold of 8: 4/8 keywords were found 1/8 utterances with false alarms

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Research Goal Applications of Research Method Experiment Demo Summary Crosscorrelation is a versatile tool for keyword spotting This was just one example of a possible algorithm Further research to optimize performance

Look Who’s Talking Now SEM Exchange, Fall 2008 October 9, Montgomery College Keyword Spotting Using Crosscorrelation Engineering Expo Banquet /08/09 Demo Research Goal Applications of Research Method Experiment Demo