2. Overview What : Stroke type Transformation: Timbre Rhythm When: Stroke timing Resynthesis

3. Video Demo

4. The Drum Dayan – treble drum Bayan – bass drum

5. DhaGeTeTeDhaGeDhaGeTeTeTe Tabla Language

6. Recognition Architecture Onset detection Statistical Model SVM Bayesian Neural Net Training data ke tun dhe ge dha te Input music Stroke Label Rhythm

7. Build Model: Training Data Several Datasets Professional musician Home recording Audio recordings manually edited and labeled

8. Build Model: Target Mapping Standardize idiosyncratic traditional naming conventions Map timbrally similar (or identical) strokes to the same category

9. Build Model: Feature Extraction Spectral Features MFCCs (24) Centroid Variance Skewness Kurtosis Slope Roll-off Variance F1 F2 F3. Fn Spectral centroid Kurtosis Feature Vector

10. Build Model: Trained Model WEKA machine learning package Support Vector Machine Models trained on different datasets can be saved for future use

11. Audio: Input Live audio is taken from a close-mic’d tabla Stereo signal provides partial separation of drums

12. Audio: Segmentation Onset detection done in Max using bonk~ More recent parallel project uses spectral flux algorithm in Java End of stroke marked by next onset (1 sec buffer size) Onset times stored

13. Audio: Feature Extraction Variance F1 F2 F3. Fn Spectral centroid Kurtosis Feature Vector

14. Output: Classification Feature vector is fed to previously trained model Single category label returned SVMlabelfeature vector

15. Output: Symbolic Score Stroke label combined with timing and amplitude information Score stored in temporary buffer in Max patch.3204.96652.3527.57156.3031.36486.3325.98276.2970.47622.3865.59281.3496.66038.7046.46211.3144.50246.7152.29906.3387.88912.2902.73426.3868.90517.3049.57271

16. Output: Timbre Remapping Stroke labels can be flexibly remapped

17. Output: Conditional Repetition

18. Output: User Interface

19. Dangum

20. Future Directions Beat tracking Modeling specific types of improvisational forms (e.g. qaida, tihai …) Automate transformations Improve interface so it can be “played” Tracking of expressive parameters (e.g. bayan pitch modulation)

21. Conclusions Shown a realtime tabla interaction system Implemented as Max java external using machine learning to identify strokes Supports flexible transformations Foundation for more general improvisation system