1. Visualizing Audio for Anomaly Detection
Mark Hasegawa-Johnson
Camille Goudeseune
Hank Kaczmarski
Thomas Huang
University of Illinois at Urbana-Champaign

2. Research Goal: Guide audio analysts to anomalies
Large dataset: audio
Anomalies
Cheap to record, expensive to play
GUI: listen 10000x faster
Robots are poor listeners, but good servants

3. Anomalies shoot down poor theories

4. Feature : audio interval  numbers
Visualization : numbers  rendering
Audio-based features (spectrogram)
Model-based features (Hnull, Hthreatening)

5. Audio-Based Features: Transformed acoustic data
Pitch, Formants
Anomaly Salience Score
Waveform A = f(t)
Spectrogram A = f(t, Hz)
Correlogram A = f(t, Hz, fundamental period)
Rate-scale representation A = f(t, Hz, bandwidth, ΔHz)
Wavelets
Multiscale

6. Model-Based Features Log-likelihood features
Display how well a hypothesis fits
Let analyst intuit a threshold of fitness
Defined by Mixture Gaussians, trained by: EM Parzen Windows One-class SVMs

7. Model-Based Features
Log-likelihood features  Log-likelihood ratios (LLR)
Because Mixture Gaussian misclassifies outliers as anomalies

8. Too many features!
Evaluate each feature with Kullback-Leibler Divergence
Combine features with AdaBoost + SVM + HMM

9. Two Interactive Testbeds
Vary features
Vary anomalies
Vary background audio
Vary how model is trained
Vary mapping from features to HSV
Anomalousness “bubbles up”

11. Multi-day audio timeline

13. 1000  μphone = The Milliphone

14. Human Subject Protocols
Tutorial
Training with immediate feedback
Measure how fast subjects find x% of anomalies

15. Influence on FODAVA Guide, don’t replace, human analysts
Guide them with zoomable features
Features from transformed data (audio-based)
Features from fitting hypotheses (model-based)

16. Developing FODAVA Make “big” audio accessible
Audio is hard, but its concepts generalize
Fast interactive exploration of time series, long (timeline) or wide (milliphone)