Christian Schmidmer, OPTICOM1 Subjective Quality Testing - Voice & Audio

Christian Schmidmer, OPTICOM2 MOS acc. To P.800 Standardized Listening Test Procedure acc. to ITU-T P.800ff Absolute Category Rating Test (ACR), no comparison to a reference signal (original) „How good does it sound?“ 5-point grading scale ‚opinion scale‘ Averaging over test subjects: MOS ‚Mean Opinion Score‘ Excellent Good Fair Poor Bad ImpairmentGrade

Christian Schmidmer, OPTICOM3 P.CONV Framework for conversational tests Two test persons having a real conversation Example scenarios (e.g. order pizza, book a flight…) … Still a bit vague, many open questions, current research.

Christian Schmidmer, OPTICOM4 P.835 Subjective assessment of noise reduction systems Similar to P.800 Separate votes for: –Speech signal –Background activity (e.g. noise) –Overall quality (high correlation with P.800 results)

Christian Schmidmer, OPTICOM5 Standardised assessment procedure for 'small impairments' in audio systems (ITU-R 1994) Comparison between reference and test signal Very sensitive to subtle distortions double-blind triple-stimulus with hidden reference Subjective Assessment in ITU-R BS.1116 OriginalAB original / coded coded / original

Christian Schmidmer, OPTICOM6 Continuous grading scale with “anchors” Subjective Difference Grade (SDG) Question: „How different do the samples sound?“ Subjective Assessment in ITU-R BS.1116

Christian Schmidmer, OPTICOM7 Subjective Testing of Intermediate Audio Quality MUSHRA - Multi Stimulus Test with Hidden Reference and Anchors  developed by EBU working group B/AIM  targets at IAQ  ITU-R BS.1534

Christian Schmidmer, OPTICOM8 MUSHRA Test Training of Subjects comparison with CD quality reference two low-pass 'anchors' (7kHz, 3.5kHz) incl. subjects can randomly access all types of codecs at similar bitrate

Christian Schmidmer, OPTICOM9 MUSHRA Test Scoring Phase subjects can randomly assess all codecs under test of similar bitrate at the same time two low-pass 'anchors' (7kHz, 3.5kHz) inc.. comparison with CD-reference, hidden reference inc.. subjects adjust slider, no score involved slider mapped to

Christian Schmidmer, OPTICOM10 Comparison of Subjective Test Methods

Christian Schmidmer, OPTICOM11 Minimum Requirements Headphones (not the cheap ones, no ear buds, no 10$ PC speakers, …) Reasonably quiet room Several listeners Good soundcard Knowledge These are the minimum requirements only, sufficient to do some prescreening. To conduct a reliable subjective test, much more efforts are required!