1. 1 Cross-Correlations and Cleaning Up Data Jessica Ferguson

2. Senior Computer Science Major English: Creative Writing Minor Pacific University

3. 3 Project Aims: HSD Project Aim 1: Collecting and transcribing spoken language data Aim 1: Collecting and transcribing spoken language data Aim 2: Automatically deriving features from spoken language samples Aim 2: Automatically deriving features from spoken language samples Aim 3: Characterizing features derived from Aim 2 Aim 3: Characterizing features derived from Aim 2

4. 4 My Task Falls under Aim 1 Falls under Aim 1 Improving the quality of the recordings in the corpus Improving the quality of the recordings in the corpus Reducing noise to give clearer speech Reducing noise to give clearer speech

5. Subjects Currently enrolled in studies at the Layton Aging and Alzheimer’s Disease Center at OHSU Individuals over 90 Individuals with Mild Cognitive Impairment (MCI)

6. 6 Test Battery Wechsler Logical Memory I/II (Story Recall) Wechsler Logical Memory I/II (Story Recall) Category Fluency (Fruits, States) Category Fluency (Fruits, States) Picture Description Task Picture Description Task Autobiographical reflections Autobiographical reflections Conversational Speech Conversational Speech

7. 7 Recording Setup Same for all sessions Same for all sessions Four different microphones set up Four different microphones set up Tests administered by examiner Tests administered by examiner

8. 8 Characteristics of Recordings Similarly-shaped waves Similarly-shaped waves Shifted horizontally Shifted horizontally

9. 9 Sample Waves

10. 10 Shifting Files Shifting files is relatively easy Shifting files is relatively easy But how far to shift? But how far to shift?

11. 11 Close-up of Comments Files

12. 12 Observed shift: Observed shift: 380 380 320 320 315 315

13. 13 Calculating Shift – Cross-Correlation Cross-correlation: a measure of how similar one signal is to another Cross-correlation: a measure of how similar one signal is to another To calculate: split the file into overlapping windows To calculate: split the file into overlapping windows Take windows of the same length in another file Take windows of the same length in another file Multiply them together Multiply them together

14. 14 Cross-Correlation Cont. The window we multiply it by in the other file keeps getting moved by one sample (1/16 msec) The window we multiply it by in the other file keeps getting moved by one sample (1/16 msec) If corresponding values have the same sign, they contribute positively If corresponding values have the same sign, they contribute positively If one is negative and the other is positive, they contribute negatively If one is negative and the other is positive, they contribute negatively We take the highest value from the range We take the highest value from the range

15. 15 Issues with Cross-Correlation With original parameters: With original parameters: Window length: 1280 samples Window length: 1280 samples Lag: -400 to 400 samples Lag: -400 to 400 samples For one value: 1280 * 800 = 512,000 For one value: 1280 * 800 = 512,000 One value every 10 msec: 100 values per second of file correlated One value every 10 msec: 100 values per second of file correlated This gets unmanageable very quickly This gets unmanageable very quickly

16. 16 Time Under Original Parameters Correlate 1.5s of files: up to 20 minutes Correlate 1.5s of files: up to 20 minutes Relatively high accuracy, but impractical Relatively high accuracy, but impractical Task: Reduce time while maintaining accuracy Task: Reduce time while maintaining accuracy

17. 17 Optimizing Parameters Parameters that could be adjusted: Parameters that could be adjusted: Window Size Window Size Lag Lag Number of correlations (how much of the file gets correlated) Number of correlations (how much of the file gets correlated)

18. 18 Window Size Initial parameters were 200 msec Initial parameters were 200 msec Decreasing below 80 msec resulted in unacceptable loss of accuracy Decreasing below 80 msec resulted in unacceptable loss of accuracy Runtime was improved but not significantly enough Runtime was improved but not significantly enough

19. 19 Number of Correlations Unfortunately, correlations are not always perfect Unfortunately, correlations are not always perfect We take the mode of the correlations produced We take the mode of the correlations produced n = 150 was the minimum, and still had a high error rate n = 150 was the minimum, and still had a high error rate

20. 20 Lag Recall the sound wave images from before: Recall the sound wave images from before:

21. 21 Lag cont. Assume that these are representative Assume that these are representative Lag values should all be between 300-400 samples (18-25 msec) Lag values should all be between 300-400 samples (18-25 msec) Add this to previous improvements: Add this to previous improvements: Runtime for one set of four files decreases to about 5-6 min Runtime for one set of four files decreases to about 5-6 min

22. 22 Other Benefits If the assumption holds: If the assumption holds: Error from optimal value decreases Error from optimal value decreases Max. error decreases from 50 msec to 6msec Max. error decreases from 50 msec to 6msec

23. 23 Original File Taken from a picture description task Taken from a picture description task

24. 24 Shifted File The same file, but correlated and shifted The same file, but correlated and shifted

25. Acknowledgements Paul Hosom and Brian Roark Fellow Interns Everyone who has made me welcome at CSLU

26. Questions?