1. ZemPod: A semantic web approach to podcasting Journal Of Web Semantics 2008 Oscar Celma, Music Technology Group, Spain Yves Raimond, Centre for Digital Music, UK August 31 th, 2009

2. Contents  Introduction  Background  System architecture  Usage scenario  Conclusions 2

3. Introduction [1/2]  Podcast  Portmanteau of the “iPod” and “broadcast”  A media file distributed in Internet  Use syndication feeds  Explosion in popularity of mobile devices  Make syndication model more attractive  Thousands of audio podcasts are available on the net 3

4. Introduction [2/2]  There are some limitations of podcasting  No formal description  Only textual description available in HTML  No information about the contents of a podcast session  Consists of a single audio file  Difficult to seek into one of the music tracks  To overcome these limitations  Using traditional audio signal processing  Speech/audio segmentation  Audio identification  Adding semantics to the podcast 4

5. Contents  Introduction  Background  Multimedia web syndication  Speech/music segmentation  Audio identification  The music ontology  System architecture  Usage scenario  Conclusions 5

6. Multimedia web syndication [1/2]  File format used for syndication  RSS  Really Simple Syndication (RSS 2.0)  Rich Site Summary (RSS 0.91 and 1.0)  RDF Site Summary (RSS 1.0)  Atom  To standardize feeds notation and autodiscovery  Due to some limitations and incompatibility versions of the RSS family 6

7. Multimedia web syndication [2/2]  Example of RSS 7

8. Feeds and the semantic web  Atom/Owl  Aims at capturing the semantics of the Atom syndication format  Feed  Attached metadata  Entry  Holds a text content 8

9. Speech/music segmentation  Discriminating between speech (or spoken content) versus music  Achieving an automatically and meaningful segmentation of a podcast session  Speech/music segmentation methods  Gaussian Mixture Models (GMM)  Support Vector Machines (SVM) classifiers  Combination of standard Hidden Markov Models and Multilayer Perceptrons 9

10. Audio identification  Allows identification of unknown music  Audio fingerprint  A unique, compact code derived from perceptually relevant aspects of a recording  Usages  Identification  Authentication  Content-based key generation  Content-based audio retrieval and processing  Hidden Markov Models (HMM)  Can precisely model temporal evolution of audio signals 10

11. Music ontology [1/2]  Create a formal framework  Describing music-related information  Covering complex editorial information  External Ontologies used by Music Ontology  OWL-Time ontology  Describing the temporal content of Web  Interval, Instant  FRBR  Functional Requirements for Bibliographic Records  Work, Expression, Manifestation, Item  FOAF  Friend Of A Friend  Person, Group, Organization 11

12. Music ontology [2/2]  Describing a music production workflow 12

13. Contents  Introduction  Background  System architecture  RDFizing a podcast session  Access and workflow  Awareness of feeds  Resource identifiers  Usage scenario  Conclusions 13

14. System architecture  Main goal is  Analysing and decomposing a given podcast audio file  RDFizing the podcast information 14

15.  The system segments the audio file into speech and music sections 15

16.  Apply speech recognition to extract a list of textual terms 16

17.  Weight terms’ relevance according to a dictionary of musical terms 17

18.  Recognize music chunks using fingerprinting 18

19.  Query a metadata repository to get basic information with the track 19

20. RDFizing a podcast session  To describe the semantics of a podcast  Using Atom-OWL and music ontology  “From 0 to 2 min, there is someone speaking about Michel Jackson, then there is a recording of a ‘Billie Jean’ in 1983”  Using 2 sub concept of the Event  MusicSegment  Temporal region holding music  SpeechSegment  Temporal region holding speech 20

21. Access and workflow  REST interface  Representational state transfer  Style of software architecture for distributed hypermedia systems such as WWW  Allow us to access the podcast service  http://zempod.net/ http://zempod.net/  Considering the podcast service is available 21

22. Access and workflow - Awareness of feeds  Internal representation of this feed  Music ontology/AtomOWL  Can be queried through SPARQL 22 USERhttp://zempod.net/feed POST 201 (Created) http://zempod.net/feed/4567 Location Identifier

23. Access and workflow - Resource identifiers  MO/AtomOWL are designed as a hierarchical URI space  Feed  Supports a syndication  http://zempod.net/feed/{FEEDID} http://zempod.net/feed/{FEEDID}  Entry  Holds a text content  http://zempod.net/feed/{FEEDID}/entry{ENTRYID} http://zempod.net/feed/{FEEDID}/entry{ENTRYID}  Item  Actual contents  http://zempod.net/feed/{FEEDID}/entry{ENTRYID}/i tem{ITEMID} http://zempod.net/feed/{FEEDID}/entry{ENTRYID}/i tem{ITEMID} 23

24. Contents  Introduction  Background  System architecture  Usage scenario  Submission of the original feed  Analysis of the new entries  Semantic description of the new entries  Conclusions 24

25. Submission of the original feed 25 http://www.ourmedia.org/u ser/billy2rivers/mrss http://zempod.net/feed POST 201 (Created) http://zempod.net/feed/1234 Location Identifier Original feed

26. Analysis of the new entries  Processing a new podcast session 26

27. Semantic description of the new entries 27 USER GET http://zempod.net/feed/1234

28. Conclusions  To solve limitations of podcasting  No formal description of a podcast  Difficult to seek into one of the music tracks  Using traditional audio signal processing  Speech/music segmentation  Audio identification  Using semantic web techniques  Transform the current RSS to the Atom/OWL  It will ease some important music information retrieval tasks 28

29. Related Ontology – MO/Event  To express the production process of a pie ce of music  The main sub-classes of event  Performance, Recording, Arrangement, Composition 29

30. Related Ontology - FRBR  Functional Requirements for Bibliographic Records  서지 레코드의 기능상 요건  목록규칙이나 목록의 완성을 의도하는 개체 - 관계 모델  서지 레코드의 구조와 관계  목록규칙 제정과 시스템 디자인을 위한 정확한 어휘 제공 30 WORKEXPRESSIONMANIFESTATIONITEM 저작표현형구현형개별자료 is realizedis embodiedis exemplified 실현되다구현되다사례가 되다 is ownedis producedis createdhas a subject 소장되다제작되다창작되다주제로 하다

31. FRBR – Entities and Relationships (1)  Entities and Primary Relationships 31

32. FRBR – Entities and Relationships (2)  Entities and “Responsibility” Relationships 32

33. FRBR – Entities and Relationships (3)  Entities and “Subject” Relationships 33 WORK MENIFESTATIO N CORPORATE BODY PERSON ITEM EXPRESSIONWORK EVENTPLACE OBJECTCONCEPT has as subject

34. MusicBrainz 34