Introduction to Music Information Retrieval (MIR) J.-S. Roger Jang (張智星) MIR Lab, CSIE Dept., National Taiwan Univ. http://mirlab.org/jang 2017/11/21

Introduction to MIR Two meanings for MIR Music Information Retrieval (音樂資訊檢索) Multimedia Information Retrieval (多媒體資訊檢索) Music IR is very different from text IR…

How to Search for a Song? Content-based search Melody Mood Genre Instrument Chords Cover songs 青山ミチ　風吹く丘で(亜麻色の髪の乙女) 島谷ひとみ / 亜麻色の髪の乙女 翻唱歌 口水歌

Types of Media for Search in MIR Quiz! Metadata-based texts  Easier Song title, artist, tags, composer, … Query inputs: text or speech Content-based info  Harder Melody, lyrics, mood, genre, chord, instruments, … Query inputs: Symbolic: notes, chord, text, … Acoustic: singing, humming, whistling, tapping, speech, recording of exact example, beatboxing… Content-based music information retrieval

Types of Acoustic Inputs for MIR Quiz! Singing/humming Query by humming (usually “ta” or “da”) Query by singing Whistling Query by whistling Normal whistle Wolf whistle Hand whistle Fingerless whistle Leaf whistle Tapping Query by tapping (at the onsets of notes) Speech Query by speech (for lyrics or meta-data) Exact but noisy example Query by example (noisy version of original clips) Beatboxing Quiz!

Types of Contents for Comparison Melody Query by humming Query by singing Query by whistling Note onsets Query by tapping (at the onsets of notes) Quiz! Metadata Query by speech Audio contents Query by examples (noisy versions of original clips) Drum patterns Query by beatboxing

Demos MIR Lab: R&D Foci and Demos

Ultimate Challenges in MIR Singing voice separation Audio melody extraction Cover song identification MP3 to MIDI conversion Approaches Deep neural networks?

Thank you for listening! Questions or comments?