1. YONSEI Univ. High Dimensional Signal Processing Lab. 1 Comparison of DSCQS and SSCQE Chulhee Lee Yonsei University

2. YONSEI Univ. High Dimensional Signal Processing Lab. 2 Test Envioroment Test tapes were made in accordance with the RRTV test plan. Subjective test using the SSCQE method was performed (23 evaluators, non-expert). From the test tapes, 64 8-second video sequences were taken and subjective test using the DSCQS was also performed (18 evaluators, non-expert) Viewing conditions: in accordance with the FRTV and RRTV test plans.

3. YONSEI Univ. High Dimensional Signal Processing Lab. 3 Detailed information for SSCQE Two 60 minutes tapes (T3, T4) which have the same PVSs in different ordering. Each 60 minutes tape has two different ordering (T312, T321, T412, T421). Number of evaluators: 6, 6, 6, 5. There are always source video sequences (SRCs) in each tape.

4. YONSEI Univ. High Dimensional Signal Processing Lab. 4 Comparison of DSCQS and SSCQE There are three subjective scores: DSCQS DMOS: FR_DMOS SSCQE MOS: RR_MOS SSCQE DMOS: RR_DMOS SSCQE_DMOS was obtained by subtracting the score of PVS from the score of the corresponding SRC.

5. YONSEI Univ. High Dimensional Signal Processing Lab. 5 FR_DMOS vs RR_MOS

6. YONSEI Univ. High Dimensional Signal Processing Lab. 6 FR_DMOS vs RR_DMOS

7. YONSEI Univ. High Dimensional Signal Processing Lab. 7 RR_DMOS vs RR_MOS

8. YONSEI Univ. High Dimensional Signal Processing Lab. 8 FR_DMOS vs PSNR

9. YONSEI Univ. High Dimensional Signal Processing Lab. 9 RR_MOS vs PSNR

10. YONSEI Univ. High Dimensional Signal Processing Lab. 10 RR_DMOS vs PSNR

11. YONSEI Univ. High Dimensional Signal Processing Lab. 11 Comparison of correlation coefficients FR_DMOSRR_MOSRR_DMOSPSNR FR_DMOS1.0000.8460.8100.766 RR_MOS0.8461.0000.8870.613 RR_DMOS0.8100.8871.0000.700 PSNR0.7660.6130.7001.000

12. YONSEI Univ. High Dimensional Signal Processing Lab. 12 RRS3 (psnr)

13. YONSEI Univ. High Dimensional Signal Processing Lab. 13 RRS4 (psnr)

14. YONSEI Univ. High Dimensional Signal Processing Lab. 14 Observations and Comments The SSCQE method is an interesting method. However, this preliminary study indicates that there exist differences between DSCQS and SSCQE. Depending on analyses, the evaluation of models might be different. Thus, there need to be agreements on how to analyze the data before the test plan is finalized. It is strongly recommended that other laboratories conduct similar comparison studies to better understand the SSCQE method.

15. YONSEI Univ. High Dimensional Signal Processing Lab. 15 Further Comments on SSCQE Attention issue: It is observed that evaluators do not record a short period of degradation. For instance, they tend to omit blocking artifacts of 1-2 seconds. Fatigue issue: It appears that 1 hour is too long for SSCQE. Variable response delay. Subjective scores tend to be more influenced by context.

16. YONSEI Univ. High Dimensional Signal Processing Lab. 16 Thank you!