ZemPod: A semantic web approach to podcasting Journal Of Web Semantics 2008 Oscar Celma, Music Technology Group, Spain Yves Raimond, Centre for Digital Music, UK August 31 th, 2009

Contents  Introduction  Background  System architecture  Usage scenario  Conclusions 2

Introduction [1/2]  Podcast  Portmanteau of the “iPod” and “broadcast”  A media file distributed in Internet  Use syndication feeds  Explosion in popularity of mobile devices  Make syndication model more attractive  Thousands of audio podcasts are available on the net 3

Introduction [2/2]  There are some limitations of podcasting  No formal description  Only textual description available in HTML  No information about the contents of a podcast session  Consists of a single audio file  Difficult to seek into one of the music tracks  To overcome these limitations  Using traditional audio signal processing  Speech/audio segmentation  Audio identification  Adding semantics to the podcast 4

Contents  Introduction  Background  Multimedia web syndication  Speech/music segmentation  Audio identification  The music ontology  System architecture  Usage scenario  Conclusions 5

Multimedia web syndication [1/2]  File format used for syndication  RSS  Really Simple Syndication (RSS 2.0)  Rich Site Summary (RSS 0.91 and 1.0)  RDF Site Summary (RSS 1.0)  Atom  To standardize feeds notation and autodiscovery  Due to some limitations and incompatibility versions of the RSS family 6

Multimedia web syndication [2/2]  Example of RSS 7

Feeds and the semantic web  Atom/Owl  Aims at capturing the semantics of the Atom syndication format  Feed  Attached metadata  Entry  Holds a text content 8

Speech/music segmentation  Discriminating between speech (or spoken content) versus music  Achieving an automatically and meaningful segmentation of a podcast session  Speech/music segmentation methods  Gaussian Mixture Models (GMM)  Support Vector Machines (SVM) classifiers  Combination of standard Hidden Markov Models and Multilayer Perceptrons 9

Audio identification  Allows identification of unknown music  Audio fingerprint  A unique, compact code derived from perceptually relevant aspects of a recording  Usages  Identification  Authentication  Content-based key generation  Content-based audio retrieval and processing  Hidden Markov Models (HMM)  Can precisely model temporal evolution of audio signals 10

Music ontology [1/2]  Create a formal framework  Describing music-related information  Covering complex editorial information  External Ontologies used by Music Ontology  OWL-Time ontology  Describing the temporal content of Web  Interval, Instant  FRBR  Functional Requirements for Bibliographic Records  Work, Expression, Manifestation, Item  FOAF  Friend Of A Friend  Person, Group, Organization 11

Music ontology [2/2]  Describing a music production workflow 12

Contents  Introduction  Background  System architecture  RDFizing a podcast session  Access and workflow  Awareness of feeds  Resource identifiers  Usage scenario  Conclusions 13

System architecture  Main goal is  Analysing and decomposing a given podcast audio file  RDFizing the podcast information 14

 The system segments the audio file into speech and music sections 15

 Apply speech recognition to extract a list of textual terms 16

 Weight terms’ relevance according to a dictionary of musical terms 17

 Recognize music chunks using fingerprinting 18

 Query a metadata repository to get basic information with the track 19

RDFizing a podcast session  To describe the semantics of a podcast  Using Atom-OWL and music ontology  “From 0 to 2 min, there is someone speaking about Michel Jackson, then there is a recording of a ‘Billie Jean’ in 1983”  Using 2 sub concept of the Event  MusicSegment  Temporal region holding music  SpeechSegment  Temporal region holding speech 20

Access and workflow  REST interface  Representational state transfer  Style of software architecture for distributed hypermedia systems such as WWW  Allow us to access the podcast service   Considering the podcast service is available 21

Access and workflow - Awareness of feeds  Internal representation of this feed  Music ontology/AtomOWL  Can be queried through SPARQL 22 USERhttp://zempod.net/feed POST 201 (Created) Location Identifier

Access and workflow - Resource identifiers  MO/AtomOWL are designed as a hierarchical URI space  Feed  Supports a syndication   Entry  Holds a text content   Item  Actual contents  tem{ITEMID} tem{ITEMID} 23

Contents  Introduction  Background  System architecture  Usage scenario  Submission of the original feed  Analysis of the new entries  Semantic description of the new entries  Conclusions 24

Submission of the original feed 25 ser/billy2rivers/mrss POST 201 (Created) Location Identifier Original feed

Analysis of the new entries  Processing a new podcast session 26

Semantic description of the new entries 27 USER GET

Conclusions  To solve limitations of podcasting  No formal description of a podcast  Difficult to seek into one of the music tracks  Using traditional audio signal processing  Speech/music segmentation  Audio identification  Using semantic web techniques  Transform the current RSS to the Atom/OWL  It will ease some important music information retrieval tasks 28

Related Ontology – MO/Event  To express the production process of a pie ce of music  The main sub-classes of event  Performance, Recording, Arrangement, Composition 29

Related Ontology - FRBR  Functional Requirements for Bibliographic Records  서지 레코드의 기능상 요건  목록규칙이나 목록의 완성을 의도하는 개체 - 관계 모델  서지 레코드의 구조와 관계  목록규칙 제정과 시스템 디자인을 위한 정확한 어휘 제공 30 WORKEXPRESSIONMANIFESTATIONITEM 저작표현형구현형개별자료 is realizedis embodiedis exemplified 실현되다구현되다사례가 되다 is ownedis producedis createdhas a subject 소장되다제작되다창작되다주제로 하다

FRBR – Entities and Relationships (1)  Entities and Primary Relationships 31

FRBR – Entities and Relationships (2)  Entities and “Responsibility” Relationships 32

FRBR – Entities and Relationships (3)  Entities and “Subject” Relationships 33 WORK MENIFESTATIO N CORPORATE BODY PERSON ITEM EXPRESSIONWORK EVENTPLACE OBJECTCONCEPT has as subject

MusicBrainz 34