Download presentation
Presentation is loading. Please wait.
1
Acoustics Research Institute Austrian Academy of Science MPEG-7 Todays Multimedia Standard Peter Balazs http://www.kfs.oeaw.ac.at Institut für Schallforschung der Österreichischen Akademie der Wissenschaften: A-1010 Wien; Liebiggasse 5. Tel. +43 1/4277-29500; Fax +43 1/4277-9296; email: xxl@kfs.oeaw.ac.at; http://www.kfs.oeaw.ac.at OeAW-ISF Peter Balazs 1999 started as programmer at the ISF 2001 finshed mathematics (University of Vienna)
2
MPEG-7 OeAW-ISF ISO / IEC Standard Mulitmedia Content Description Interface Multimedia data / metadata description system Low Level – High Level; content based Open system Inheritance Description of methods normativ – informativ
3
MPEG-7 OeAW-ISF ISO / IEC Standard Mulitmedia Content Description Interface Multimedia data / metadata description system Low Level – High Level Open system Inheritance Description of methods normativ – informativ IDDogBarks IDState1 0.000 IDState2 0.000 IDState3 0.045 IDState4 0.000 IDState5 0.442 IDState6 0.513
4
MPEG-7 OeAW-ISF History Call for Proposals October 1998 Evaluation February 1999 First version of Working Draft (WD) December 1999 Committee Draft (CD) October 2000 Final Committee Draft (FCD) February 2001 Final Draft International Standard (FDIS) July 2001 International Standard (IS) September 2001 Development Amendment AudioMay 2002 Call for Proposals (Systems, version 2)July 2002 MPEG 21 international standardApril 2009
5
XML = eXtensible Markup Language XML OeAW-ISF Metasprache Hypertext Markup markup = tag... Open Standard <!DOCTYPE document [ <!ELEMENT ADRESSE (Vorname, Nachname, Wohnort)>.... ]> <!DOCTYPE document [ <!ELEMENT ADRESSE (Vorname, Nachname, Wohnort)>.... ]> Peter Balazs Tulln........ <!DOCTYPE document [ <!ELEMENT ADRESSE (Vorname, Nachname, Wohnort)>.... ]> Peter Balazs Tulln........
6
XML = eXtensible Markup Language XML OeAW-ISF Metasprache Hypertext Markup markup = tag... Open Standard <!DOCTYPE document [ <!ELEMENT ADRESSE (Vorname, Nachname, Wohnort)>.... ]> Peter Balazs Tulln........ CursorOpts = 0 0 1 440 SignalOpts= 1 1 FrameOpts= 40 1 75 2 0 1 GraphXY= 0 1e4 1 -80 50 1 Method= 0 32 20 0 1 0 0 0 1 0 0 Average= 0 0 99
7
MPEG-7 OeAW-ISF Descriptors Low Level Descriptor Schemes High Level, container Descriptor Definition Language (DDL) XML Schema, STX Schema System Tools ASCII Text - binary
8
MPEG-7 OeAW-ISF Out of [1]
![MPEG-7 OeAW-ISF Out of [1]](http://images.slideplayer.com./3/915480/slides/slide_8.jpg "MPEG-7 OeAW-ISF Out of [1]")
9
OeAW-ISF MPEG-7 Audio: Low Level Descriptors Single Sample Segments DS, compare to STX Out of [1]
![OeAW-ISF MPEG-7 Audio: Low Level Descriptors Single Sample Segments DS, compare to STX Out of [1]](http://images.slideplayer.com./3/915480/slides/slide_9.jpg "OeAW-ISF MPEG-7 Audio: Low Level Descriptors Single Sample Segments DS, compare to STX Out of [1]")
10
OeAW-ISF MPEG-7 Audio: Low Level Descriptors Scalar Vector Single Series series of vectors = table, matrix Scalable Series Out of [2]
![OeAW-ISF MPEG-7 Audio: Low Level Descriptors Scalar Vector Single Series series of vectors = table, matrix Scalable Series Out of [2]](http://images.slideplayer.com./3/915480/slides/slide_10.jpg "OeAW-ISF MPEG-7 Audio: Low Level Descriptors Scalar Vector Single Series series of vectors = table, matrix Scalable Series Out of [2]")
11
OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower
12
OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness
13
OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness Signal Parameters AudioHarmonicity, AudioFundamentalFrequency
14
OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness Signal Parameters AudioHarmonicity, AudioFundamentalFrequency Timbral Temporal LogAttackTime, TemporalCentroid
15
OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness Signal Parameters AudioHarmonicity, AudioFundamentalFrequency Timbral Temporal LogAttackTime, TemporalCentroid Timbral Spectral SpectralCentroid, HarmonicSpectralCentroid, HarmonicSpectralDeviation, HarmonicSpectralSpread, HarmonicSpectralVariation
16
OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness Spectral Basis AudioSpectrumBasis, AudioSpectrumProjection Signal Parameters AudioHarmonicity, AudioFundamentalFrequency Timbral Temporal LogAttackTime, TemporalCentroid Timbral Spectral SpectralCentroid, HarmonicSpectralCentroid, HarmonicSpectralDeviation, HarmonicSpectralSpread, HarmonicSpectralVariation Out of [1]
![OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness Spectral Basis AudioSpectrumBasis, AudioSpectrumProjection Signal Parameters AudioHarmonicity, AudioFundamentalFrequency Timbral Temporal LogAttackTime, TemporalCentroid Timbral Spectral SpectralCentroid, HarmonicSpectralCentroid, HarmonicSpectralDeviation, HarmonicSpectralSpread, HarmonicSpectralVariation Out of [1]](http://images.slideplayer.com./3/915480/slides/slide_16.jpg "OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness Spectral Basis AudioSpectrumBasis, AudioSpectrumProjection Signal Parameters AudioHarmonicity, AudioFundamentalFrequency Timbral Temporal LogAttackTime, TemporalCentroid Timbral Spectral SpectralCentroid, HarmonicSpectralCentroid, HarmonicSpectralDeviation, HarmonicSpectralSpread, HarmonicSpectralVariation Out of [1]")
17
OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness Spectral Basis AudioSpectrumBasis, AudioSpectrumProjection Signal Parameters AudioHarmonicity, AudioFundamentalFrequency Timbral Temporal LogAttackTime, TemporalCentroid Timbral Spectral SpectralCentroid, HarmonicSpectralCentroid, HarmonicSpectralDeviation, HarmonicSpectralSpread, HarmonicSpectralVariation Out of [1]
![OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness Spectral Basis AudioSpectrumBasis, AudioSpectrumProjection Signal Parameters AudioHarmonicity, AudioFundamentalFrequency Timbral Temporal LogAttackTime, TemporalCentroid Timbral Spectral SpectralCentroid, HarmonicSpectralCentroid, HarmonicSpectralDeviation, HarmonicSpectralSpread, HarmonicSpectralVariation Out of [1]](http://images.slideplayer.com./3/915480/slides/slide_17.jpg "OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness Spectral Basis AudioSpectrumBasis, AudioSpectrumProjection Signal Parameters AudioHarmonicity, AudioFundamentalFrequency Timbral Temporal LogAttackTime, TemporalCentroid Timbral Spectral SpectralCentroid, HarmonicSpectralCentroid, HarmonicSpectralDeviation, HarmonicSpectralSpread, HarmonicSpectralVariation Out of [1]")
18
OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness Spectral Basis AudioSpectrumBasis, AudioSpectrumProjection Signal Parameters AudioHarmonicity, AudioFundamentalFrequency Timbral Temporal LogAttackTime, TemporalCentroid Timbral Spectral SpectralCentroid, HarmonicSpectralCentroid, HarmonicSpectralDeviation, HarmonicSpectralSpread, HarmonicSpectralVariation Out of [2] Silence Out of [1]
![OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness Spectral Basis AudioSpectrumBasis, AudioSpectrumProjection Signal Parameters AudioHarmonicity, AudioFundamentalFrequency Timbral Temporal LogAttackTime, TemporalCentroid Timbral Spectral SpectralCentroid, HarmonicSpectralCentroid, HarmonicSpectralDeviation, HarmonicSpectralSpread, HarmonicSpectralVariation Out of [2] Silence Out of [1]](http://images.slideplayer.com./3/915480/slides/slide_18.jpg "OeAW-ISF MPEG-7 Audio: Low Level Descriptors Basic AudioWaveform, AudioPower Basic Spectral AudioSpectrumEnvelope, AudioSpectrumCentroid, AudioSpectrumSpread, AudioSpectrumFlatness Spectral Basis AudioSpectrumBasis, AudioSpectrumProjection Signal Parameters AudioHarmonicity, AudioFundamentalFrequency Timbral Temporal LogAttackTime, TemporalCentroid Timbral Spectral SpectralCentroid, HarmonicSpectralCentroid, HarmonicSpectralDeviation, HarmonicSpectralSpread, HarmonicSpectralVariation Out of [2] Silence Out of [1]")
19
OeAW-ISF MPEG-7 Audio: High Level DSs AudioSignature AudioSpectrumFlatness
20
OeAW-ISF MPEG-7 Audio: High Level DSs AudioSignature AudioSpectrumFlatness Musical Instrument Timbre Description Tool HarmonicInstrumentTimbre (LAT + timbre spectral) PercussiveInstrumentTimbre (timbre temporal + SpectralCentroid)
21
OeAW-ISF MPEG-7 Audio: High Level DSs AudioSignature AudioSpectrumFlatness Musical Instrument Timbre Description Tool HarmonicInstrumentTimbre (LAT + timbre spectral) PercussiveInstrumentTimbre (timbre temporal + SpectralCentroid) Melody Description Tools MelodyContour DS, Melody Sequence DS
22
OeAW-ISF MPEG-7 Audio: High Level DSs AudioSignature AudioSpectrumFlatness Musical Instrument Timbre Description Tool HarmonicInstrumentTimbre (LAT + timbre spectral) PercussiveInstrumentTimbre (timbre temporal + SpectralCentroid) Melody Description Tools MelodyContour DS, Melody Sequence DS General Sound Recognition and Indexing Description Tool SpectralBasis, SoundClassificationModel : SoundModels, classification scheme; SoundModelStatePath, SoundModelStateHistogram
23
OeAW-ISF MPEG-7 Audio: High Level DSs AudioSignature AudioSpectrumFlatness Musical Instrument Timbre Description Tool HarmonicInstrumentTimbre (LAT + timbre spectral) PercussiveInstrumentTimbre (timbre temporal + SpectralCentroid) Melody Description Tools MelodyContour DS, Melody Sequence DS General Sound Recognition and Indexing Description Tool SpectralBasis, SoundClassificationModel : SoundModels, classification scheme; SoundModelStatePath, SoundModelStateHistogram SpokenContentDescription Tools SpokenContentHeader : WordLexicon, PhonLexicon; SpokenContentLattice: WordLinks, PhonLinks.
24
OeAW-ISF MPEG-7 Audio: Amendment New Base types optional attribute for channel Modification of Spoken Content Description Tools acoustics only score possible for speech recognition; prosody, syllabels Audio Signal Quality DS BackgroundNoiseLevel, BalanceType, DCoffsetType, BandwidthType. TransmissionTechnologyType: shellac, vinyl,.... Additional Tools: tempo description, compact variable precision representation (BAM) Liguistic Description Tools: semantic structure of liguistic data
25
OeAW-ISF MPEG-7 Literatur: [1] José M. Martínez, MPEG-7 Overview (version 8) ISO/IEC JTC1/SC29/WG11N4980, Klagenfurt, July 2002, http://mpeg.telecomitalialab.com/standards/mpeg-7/mpeg-7.htmhttp://mpeg.telecomitalialab.com/standards/mpeg-7/mpeg-7.htm [2] ISO / IEC, Information Technology – Multimedia Content Description Interface – Part 4: Audio, Geneva, July 2001 [3] Oliver Pott, Günter Wielange, XML Praxis und Referenz, München 2001 [4] J. Bitzer, J. H. Martínez, Information Technology Multimedia Content Description Interface Part 4: Audio Proposed Draft Amendment, Fairfax, May 2002 Links: [4] MPEG Home Page, http://mpeg.telecomitalialab.com/http://mpeg.telecomitalialab.com/ [5] Extensible Markup Language, http://www.w3.org/XML/http://www.w3.org/XML/ [6] STX, http://www.kfs.oeaw.ac.at/software.htmhttp://www.kfs.oeaw.ac.at/software.htm
![OeAW-ISF MPEG-7 Literatur: [1] José M.](http://images.slideplayer.com./3/915480/slides/slide_25.jpg "Martínez, MPEG-7 Overview (version 8) ISO/IEC JTC1/SC29/WG11N4980, Klagenfurt, July 2002, [2] ISO / IEC, Information Technology – Multimedia Content Description Interface – Part 4: Audio, Geneva, July 2001 [3] Oliver Pott, Günter Wielange, XML Praxis und Referenz, München 2001 [4] J. Bitzer, J. H. Martínez, Information Technology Multimedia Content Description Interface Part 4: Audio Proposed Draft Amendment, Fairfax, May 2002 Links: [4] MPEG Home Page, [5] Extensible Markup Language, [6] STX,")
Similar presentations
