Joint Effort Group Ghent meeting A. Bourret P. Le Callet

Goals Scientific exchange not only limited to subjective quality assessment –Links with other communities who have the opposite problem : share on objective metric not on subjective Context : –MPEG demanding –JPEG AIC initiative

Joint Effort approach Starting with generic model framework Testing only new modules' benefit Source code available –no training possibility –no need for secret sequences –each CE member can run its own tests and report results better understanding of each new algo benefits

Policies, IPR issues Among possibilities JVT (H264 …) example : –Contribution and IPR declaration and statement at the same time –Contribution Form : source code Issue: reference software –Patent pool

Policies : dataset Dataset (e.g. materials and subjective data) … two steps : –Common Working Dataset WD (known by contributors) to test the contributions Possibility to present analysis based on non public dataset –Goal : identify limits of contribution in order to further improvement –Validation dataset VD at final stage (joined the WD after)

Process example Step 1:Scope (limited), rules (performance evaluation), basic framework, call for Evidence Step 2: WD completed Step 3: contributions and evaluation on WD => decision (JEG labs) Step 4: VD design and evaluation T0 + 3 T0 + 9 T T0 + 27

Possible Scope Previous test plan Current Non existing

Profiles and levels Name (NR, RR, Hybrid..): VQM1-NR –Profile: use and context (coding errors, tranmission errors, accuracy and quality range) Level: resolution, frame rate, coder … Name: VQM1-H264 –Prolile: NR, RR, FR, Hybrid Level: resolution, …

First test plan proposal VQM1-H264 –Profile: NR V (PVS only after decoding) S (Stream only) H (PVS and Stream) At least : Stream at NAL level Possibility to define extended profile to cover protocol level parameters (UDP, RTP, DVB …) –Level: from HD to QCIF, transmission error at several bit rate

VQM1-H264 NRS profile NRV profile NRH profile Tools that have access to PVS only Tools that have access to stream only