Music Information Retrieval from a Singing Voice Using Lyrics and Melody Information Motoyuki Suzuki, Toru Hosoya, Akinori Ito, and Shozo Makino EURASIP Journal on Advances in Signal Processing, Volume 2007 presenter : 王崇喆

2 Outline Overview of the system Lyrics recognition based on a finite state automaton Verification of hypothesis using melody information Experiments Discussion

3 Overview of the system Each hypothesis h has the following information Song name S(h) Recognized text W(h) Recognition score R(h) Time alignment information F(h)

4 Lyrics recognition based on a finite state automaton Acoustic model was trained from read speech MLLR (maximum likelihood linear regression) method was used as an adaptation algorithm.

5 Verification of hypothesis using melody information The melody information (relative pitch/IOI of each note) can be calculated from the tune in the database extracted using the estimated pitch sequence of the singing voice and time alignment information Pitch sequence is calculated by the praat system frame-by-frame The pitch of the note is defined as the median of the pitch sequence corresponding to the note IOI of the note is obtained as the duration between boundaries.

6 Experiments(1/2) The number of songs in the database was 156 Test queries : singing voice, each of which consisted of 5 words Singers : 6 male university students 110 choruses were collected as song data The total number of test queries was hypotheses were output from the lyrics recognizer Some similar hypotheses were output as another hypotheses W(h) is slightly different from W(h), even though S(h) is exactly the same as S(h) The maximum retrieval accuracy was limited to 97.4%

7 Experiments(2/2)

8 Discussion The proposed system assumes that the input singing voice consists of a part of the correct lyrics the lyrics recognizer can correctly recognize because of the grammatical restriction of FSA. The proposed system was examined using a very small database. When the system is applied to practical use, following 2 problems will be occurred computation time in the lyrics recognition step pre-selection algorithm would be needed before lyrics recognition deterioration of the recognition performance There are many songs which have similar lyrics in the large database these misrecognition can be corrected by using melody information

9 The End \ ⊙▽⊙ /