Overview What : Stroke type Transformation: Timbre Rhythm When: Stroke timing Resynthesis

Video Demo

The Drum Dayan – treble drum Bayan – bass drum

DhaGeTeTeDhaGeDhaGeTeTeTe Tabla Language

Recognition Architecture Onset detection Statistical Model SVM Bayesian Neural Net Training data ke tun dhe ge dha te Input music Stroke Label Rhythm

Build Model: Training Data Several Datasets Professional musician Home recording Audio recordings manually edited and labeled

Build Model: Target Mapping Standardize idiosyncratic traditional naming conventions Map timbrally similar (or identical) strokes to the same category

Build Model: Feature Extraction Spectral Features MFCCs (24) Centroid Variance Skewness Kurtosis Slope Roll-off Variance F1 F2 F3. Fn Spectral centroid Kurtosis Feature Vector

Build Model: Trained Model WEKA machine learning package Support Vector Machine Models trained on different datasets can be saved for future use

Audio: Input Live audio is taken from a close-mic’d tabla Stereo signal provides partial separation of drums

Audio: Segmentation Onset detection done in Max using bonk~ More recent parallel project uses spectral flux algorithm in Java End of stroke marked by next onset (1 sec buffer size) Onset times stored

Audio: Feature Extraction Variance F1 F2 F3. Fn Spectral centroid Kurtosis Feature Vector

Output: Classification Feature vector is fed to previously trained model Single category label returned SVMlabelfeature vector

Output: Symbolic Score Stroke label combined with timing and amplitude information Score stored in temporary buffer in Max patch

Output: Timbre Remapping Stroke labels can be flexibly remapped

Output: Conditional Repetition

Output: User Interface

Dangum

Future Directions Beat tracking Modeling specific types of improvisational forms (e.g. qaida, tihai …) Automate transformations Improve interface so it can be “played” Tracking of expressive parameters (e.g. bayan pitch modulation)

Conclusions Shown a realtime tabla interaction system Implemented as Max java external using machine learning to identify strokes Supports flexible transformations Foundation for more general improvisation system