Adapted representations of audio signals for music instrument recognition Pierre Leveau Laboratoire d’Acoustique Musicale, Paris - France GET - ENST (Télécom Paris), France

Pierre Leveau - ENST - LAM2 Summary Master Thesis: Music instrument recognition on solo performances with signal segmentation (transient part / release part) Ph. D. Thesis: Structured and sparse decompositions: application to audio indexing

Pierre Leveau - ENST - LAM3 Music Instrument Recognition Basic Scheme Feature extraction Training DB (manually indexed) Classification model Comparison to the model File to analyze Feature extraction decision Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM4 Feature Extraction Feature Extraction on frames of fixed size (30 ms) Analysis Frames Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM5 Music Note Scheme time energy Ex: strong attack instrument Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM6 Interest of transients for Music Instrument Recognition pianotrumpet cello flute Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM7 Chosen Method Signal segmentation into transient part / release part Approximation: fixed length transients Need of an automatic onset detection algorithm. Study of solo performances Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM8 Onset Detection Detection function (ex: high frequency content, spectral difference, phase deviation…) Peak-picking Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM9 Evaluation of Onset Detection Necessity of an reference onset database ROC Curves good detections % false alarms % Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM10 Sound Onset Labelization spectrogram Signal plot Sound listening and labels positioning Reference Onset and Sound Databases Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM11 Onset Database Annotation precision depending on the file type Detection function evaluation must take it into account Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM12 Annotation precision: examples trumpet cello Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM13 Developed Detection Function Complex Spectral Difference: Delta Complex Spectral Difference: guitarviolin Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM14 Detection Function comparison Tolerance window T ROC = 100 ms T ROC = T opt Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM15 Signal segmentation RRTTT R R T Analysis Frames Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM16 Music Instrument recognition on transients - Results Music instrument recognition only on transients implies: - big decrease of the learning database size - for a fixed duration of the test signal, less data to take a decision. Results worse than for a recognition on all frames Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM17 Perspectives Increase the onset database size for a more robust evaluation Improve the robustness of the Onset detection algorithm Merge decisions on transients and steady part, compare to the classical static recognition. Select features adapted for each part of the notes. Music instrument recognition on solo performances with signal segmentation (transient part / release part)

Pierre Leveau - ENST - LAM18 Ph. D. Thesis Subject: Sparse and structured decompositions: application to audio indexing Under supervision of Gaël Richard (GET - ENST, Paris) and Laurent Daudet (Laboratoire d’Acoustique Musicale, Paris)

Pierre Leveau - ENST - LAM19 Sparse Representations Classical representations: Orthogonal transform (ex: Fourier Transform, STFT, MDCT, Wavelet Transform…) Redundant representations: Sparse representations (only on N terms): : Redundant dictionnary Sparse and structured decompositions: application to audio indexing

Pierre Leveau - ENST - LAM20 Dictionary Example C: MDCT basis (useful to represent tonal parts of signals) W: DWT basis (useful to represent transient parts of signals) Sparse and structured decompositions: application to audio indexing

Pierre Leveau - ENST - LAM21 Algorithms Matching Pursuit (and its variants):  Greedy algorithms  Based on an iterative search  Faster algorithm needs a suboptimal search Molecular Matching Pursuit:  Gives structured, perceptually relevant organizations of the atoms (by grouping significant coefficients)  Faster than standard MP  Fast varying frequencies (ex: vibrato) cannot be efficiently represented Sparse and structured decompositions: application to audio indexing

Pierre Leveau - ENST - LAM22 Application to music instrument recognition Signal Feature Extraction Classical Music Instrument Recognition Comparison to statistical models Decision Signal MMP Feature Extraction (which features?) Comparison to statistical models (which models?) Decision Music Instrument Recognition with sparse decomposition features Structured Representation Sparse and structured decompositions: application to audio indexing

Pierre Leveau - ENST - LAM23 To be continued… Thank you for your attention.