Berenzweig and Ellis - WASPAA 011 Locating Singing Voice Segments Within Music Signals Adam Berenzweig and Daniel P.W. Ellis LabROSA, Columbia University

Berenzweig and Ellis - WASPAA 012 LabROSA What Where Who Why you love us

Berenzweig and Ellis - WASPAA 013 The Future as We Hear It Online Digital Music Libraries The Coming Age of Streaming Music Services Information Retrieval: How do we find what we want? Recommendation: How do we know what we want to find? –Collaborative Filtering vs. Content-Based –What is Quality?

Berenzweig and Ellis - WASPAA 014 Motivation Lyrics Recognition: Baby Steps –Segmentation –Forced Alignment –A Corpus Song structure through singing structure? –Fingerprinting –Retreival –Feature for similarity measures

Berenzweig and Ellis - WASPAA 015 Lyrics Recognition: Can YOU do it? Notoriously hard, even for humans. –amIright.com, kissThisGuy.com Why so hard? –Noise, music, whatever. –Singing is not speech: voice transformations –Strange word sequences (“poetry”) Need a corpus

Berenzweig and Ellis - WASPAA 016 History of the Problem Segmentation for Speech Recognition: Music/Speech –Scheirer & Slaney Forced Alignment - Karaoke –Cano et al. [REF NEEDED] Acoustic feature design: Custom job or Kitchen Sink? Idea! Use a speech recognizer: PPF (Posterior Probability Features) –Williams & Ellis Ultimately: Source separation, CASA

Berenzweig and Ellis - WASPAA 017 A Peek at the End

Berenzweig and Ellis - WASPAA 018 Architecture Overview Audio PLP Speech Recognizer (Neural Net) Feature Calculation posteriogramcepstra Time- averaging Entropy H H /h# Dynamism D P(h#) Segmentation (HMM) Gaussian Model Gaussian Model

Berenzweig and Ellis - WASPAA 019 Architecture Overview Audio PLP Speech Recognizer (Neural Net) posteriogramcepstra Segmentation (HMM) Neural Net Neural Net

Berenzweig and Ellis - WASPAA 0110 “So how’s that working out for you, being clever?” Entropy Entropy excluding background Dynamism Background probability Distribution Match: Likelihoods under single Gaussian model –Cepstra –PPF

Berenzweig and Ellis - WASPAA 0111 Recovering context with the HMM Transition probabilities –Inverse average segment duration Emission probabilities –Gaussian fit to time-averaged distribution Segmentation: the Viterbi path Evaluation –Frame error rate (no boundary consideration)

Berenzweig and Ellis - WASPAA 0112 Results [Table, figures] Listen! –Good, bad –trigger & stick –genre effects?

Berenzweig and Ellis - WASPAA 0113 Results

Berenzweig and Ellis - WASPAA 0114 E =.075 P(h#) in effect

Berenzweig and Ellis - WASPAA 0115 E =.68 P(h#) gone bad

Berenzweig and Ellis - WASPAA 0116 E =.61 Strong phones trigger, but can’t hold it Production quality effect? ‘ey’ ‘uw’ ‘m’,’n’

Berenzweig and Ellis - WASPAA 0117 E =.25 “Trigger and Stick” ‘s’

Berenzweig and Ellis - WASPAA 0118 E =.54 False phones ‘bcl’,’dcl’, ’b’, ‘d’ ‘l’,’r’

Berenzweig and Ellis - WASPAA 0119 E =.20 Genre effect?

Berenzweig and Ellis - WASPAA 0120 Discussion The Moral of the Story: Just give it the data PPF is better than cepstra. Speech Recognizer is pretty powerful. Why does the extra Gaussian model help PPF but not cepstra? Time averaging helps PPF: proves that it’s using the overall distribution, not short-time detail (at least, when modelled by single gaussians)