1. An XML Multi-layer Framework for Music Information Description
Luca A. Ludovico
LIM (Laboratorio di Informatica Musicale)
DICo (Dipartimento di Informatica e Comunicazione)
Università degli Studi di Milano
Via Comelico, Milano, Italia

2. Motivations and State of the Art
Part I
Motivations and State of the Art
Part I: Motivations and State of the Art
Part II: Methodology of Work
Part III: The MX Format
Part IV: Scientific Advancement
Part V: Conclusions

3. An XML Multi-layer Framework for Music Information Description
Music contents
Music contents are heterogeneous:
Symbolic (scores from a logic standpoint)
General (catalog metadata)
Structural (relationships among music objects)
Notational (graphical instances)
Computer-driven performances (computer generated music)
Audio/video (recordings)
Other aspects (sketches, fashion plates, on-stage photos, playbills, etc.)
15/01/2019
An XML Multi-layer Framework for Music Information Description

4. Existing formats Music contents and corresponding formats
Symbolic DARMS, NIFF, MusicXML, …
General TXT, XML, database, …
Structural [?]
Notational EPS, JPEG, TIFF, …
Computer-driven performances Csound, MIDI, SASL/SAOL, …
Audio/video AIFF, MP3, WAV; AVI, MPEG, …
Other EPS, JPEG, TIFF, …
Possibility to describe only particular aspects of music information
15/01/2019
An XML Multi-layer Framework for Music Information Description

5. An XML Multi-layer Framework for Music Information Description
State of the art
Close, proprietary and non-standard formats
Often binary (non-readable) formats
Many formats to catch different aspects of music information
No way to synchronize heterogeneous music information described in different formats
15/01/2019
An XML Multi-layer Framework for Music Information Description

6. Part II Methodology of Work Part I: Motivations and State of the Art
Part III: The MX Format
Part IV: Scientific Advancement
Part V: Conclusions

7. An XML Multi-layer Framework for Music Information Description
Steps
Analysis of the problem from the point of view of
Music
Information Technology
Communication
Interaction with experts and other workgroups
Choice of a technological framework
Definition of a suitable format
Validation
15/01/2019
An XML Multi-layer Framework for Music Information Description

8. Part III The MX Format Part I: Motivations and State of the Art
Part II: Methodology of Work
Part III: The MX Format
Part IV: Scientific Advancement
Part V: Conclusions

9. An XML Multi-layer Framework for Music Information Description
The MX format
MX is an XML-based format to achieve a comprehensive description of music information
Key requirements
Multimedia support
Support for in-use formats
Distinguishing Features
Multi-layer structuring of (heterogeneous) music information
Full synchronization among contents
15/01/2019
An XML Multi-layer Framework for Music Information Description

10. Multi-layer structuring
Music information is heterogeneous
Consequences on the format:
From a qualitative standpoint, heterogeneous information should be supported
From a quantitative standpoint, more objects of the same type should be supported
Common structure referenced by all the layers: spine
15/01/2019
An XML Multi-layer Framework for Music Information Description

11. An XML Multi-layer Framework for Music Information Description
MX layers
General
Music Logic
Layers
Structural
Notational
Performance
Audio
15/01/2019
An XML Multi-layer Framework for Music Information Description

12. An XML Multi-layer Framework for Music Information Description
MX layers in detail
General - catalog metadata
Logic
Spine
Logically Organized Symbols (LOS)
Layout
Structural - musicological analyses, relationships among music objects
Notational - graphical aspect of score
Performance - sub-symbolic formats for computer-driven performances
Audio - audio formats
General
Logic
Structural
Notational
Performance
Audio
15/01/2019
An XML Multi-layer Framework for Music Information Description

13. An XML Multi-layer Framework for Music Information Description
Spine
Definition: a space-time construct
In practice: music event identification
Spacing and timing: virtual and relative
Purposes:
Glue among layers
Synchronization
15/01/2019
An XML Multi-layer Framework for Music Information Description

14. An XML Multi-layer Framework for Music Information Description
15/01/2019
An XML Multi-layer Framework for Music Information Description

15. Intra-layer synchronization
It takes place among objects belonging to the same layer: homogeneous information
15/01/2019
An XML Multi-layer Framework for Music Information Description

16. Inter-layer synchronization
It takes place among different layers: heterogeneous information
15/01/2019
An XML Multi-layer Framework for Music Information Description

17. How synchronization is achieved
15/01/2019
An XML Multi-layer Framework for Music Information Description

18. Part IV: Scientific Advancement
Part I: Motivations and State of the Art
Part II: Methodology of Work
Part III: The MX Format
Part IV: Scientific Advancement
Part V: Conclusions

19. An XML Multi-layer Framework for Music Information Description
Thesis contributions
Definition of a reference unified framework for XML-based music specification:
integrated with pre-existing partial formats
multi-layered around the spine core element
incorporating synchronization capabilities
15/01/2019
An XML Multi-layer Framework for Music Information Description

20. An XML Multi-layer Framework for Music Information Description
Synchronization
Analysis of different facets of music information.
Design and implementation of XML descriptors for the synchronization of heterogeneous contents.
Design and implementation of algorithms to allow advanced music enjoyment.
15/01/2019
An XML Multi-layer Framework for Music Information Description

21. An XML Multi-layer Framework for Music Information Description
Contribution to IEEE P1599
Institute of Electrical and Electronics Engineers (IEEE) Computer Society
Task Force on Computer Generated Music
Technical Committee on Computer Generated Music
Project Authorization Request 1599 (IEEE P1599): “Definition of a Commonly Acceptable Musical Application Using the XML Language”
“MAX2002 – First International Conference on Musical Application using XML”
2006 – PhD thesis: “An XML Multi-layer Framework For Music Information Description”
March 2007 – Balloting
15/01/2019
An XML Multi-layer Framework for Music Information Description

22. MX-based Applications
Three ways to enjoy music contents
Simple synchronized fruition
Real-time switch among multimedia contents
User-interaction with immediate synchronization
An example: MX Navigator
15/01/2019
An XML Multi-layer Framework for Music Information Description

23. List of MX-related papers
HAUS, Goffredo, e LUDOVICO, Luca A., Music Segmentation: an XML-Oriented Approach, CMMR 2004 Post-Symposium Proceedings, Lecture Notes in Computer Science (LNCS), Springer Verlag, 2004
CASTANO, Silvana, FERRARA, Alfio, HAUS, Goffredo, LUDOVICO, Luca Andrea, MONTANELLI, Stefano, RACCA, Gianpaolo, e VERCELLESI, Giancarlo, The MX Formalism for Semantic Web-Compatible Representation of Music Metadata, International Workshop on Multimedia Information Systems (MIS) 2005 Proceedings, Lecture Notes in Computer Science (LNCS), Springer Verlag, 2005
BARATE', Adriano, HAUS, Goffredo, LUDOVICO, Luca Andrea,  e VERCELLESI, Giancarlo, MXDemo: a Case Study about Audio, Video, and Score Synchronization, AXMEDIS 2005 Proceedings, Firenze, 2005
BAGGI, Denis, BARATE', Adriano, HAUS, Goffredo, e LUDOVICO, Luca A., A Computer Tool to Enjoy and Understand Music, EWIMT 2005 Workshop Proceedings, London, 2005
CASTANO, Silvana, FERRARA, Alfio, HAUS, Goffredo, LUDOVICO, Luca Andrea, MONTANELLI, Stefano, A Semantic Web Ontology for Context-based Classification and Retrieval of Music Resources, ACM Transactions on Multimedia Computing, Communications, and Applications.
BARATE', Adriano, HAUS, Goffredo, e LUDOVICO, Luca A., An XML-Based Format for Advanced Music Fruition, Sound and Music Computing Conference 2006 (SMC 2006), Marseille.
BAGGI, Denis, BARATE', Adriano, HAUS, Goffredo, e LUDOVICO, Luca A., Developing Intuition in Engineering Education: New Technology To Capture Structures in Music, 35th International IGIP Symposium, Tallinn, 2006.
BARATE', Adriano, HAUS, Goffredo, e LUDOVICO, Luca A., MX Navigator: An Application for Advanced Music Fruition, AXMEDIS 2006, Leeds, 2006.
BARATE', Adriano, HAUS, Goffredo, e LUDOVICO, Luca A., Music representation of score, sound, MIDI, structure and metadata all integrated in a single multilayer environment based on XML, Intelligent Music Information Systems, IRM Press, 2006
BARATE', Adriano, e LUDOVICO, Luca A., An XML-based Synchronization of Audio and Graphical Representations of Music Scores, Proceedings of the 8th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2007), Santorini, Greece, 2007
15/01/2019
An XML Multi-layer Framework for Music Information Description

24. Part IV: Conclusions Part I: Motivations and State of the Art
Part II: Methodology of Work
Part III: The MX Format
Part IV: Scientific Advancement
Part V: Conclusions

25. An XML Multi-layer Framework for Music Information Description
Recent work
New elements and attributes to catch other kinds of description, models, forms of analysis, etc.
Implementation of an SDK in C# aimed at MX encoding
Implementation of an MX integrated editor/viewer
IEEE Balloting
15/01/2019
An XML Multi-layer Framework for Music Information Description

26. Further research issues
Integration of ontologies for music information retrieval inside General layer [Ferrara et al. 2005]
Automatic translation among layers and automatic generation of synchronization information
Automatic segmentation
Automatic extraction of information
15/01/2019
An XML Multi-layer Framework for Music Information Description

27. An XML Multi-layer Framework for Music Information Description
Questions
Thank you for your attention!
15/01/2019
An XML Multi-layer Framework for Music Information Description