1. Performance metrics and configuration strategies for group network communication Tom Z. J. FU Dah Ming Chiu John C. S. Lui

2. Outline  Motivation  GMOS  Leader Selection Strategy  Two-step Mapping Method  Experiment setup and results  Applications and Conclusion

3. Motivation 1.Existing methods are designed for two-party communication. 2.Number of group-based multimedia applications is increasing. 3.Need appropriate metrics to evaluate QoS of group multimedia session. 4.Help select the proper strategy to distribute streams or mixed data to provide better QoS.

4. Group Mean Opinion Score GMOS metric evaluates the overall quality of the multi-party multimedia session. A B C U MOS A MOS B MOS C GMOS GMOS = f (MOS A, MOS B, MOS C )

5. Group Mean Opinion Score Define:

6. GMOS The range and meaning of GMOS:

7. GMOS experiment results We invite subjects to do experiments and find the distribution of parameter α. 1.25 subjects are invited 2.18 conference records 3.Subject gives two kinds of scores 4.437 MOSes and GMOS pairs 5.392α are determined

8. GMOS experiment results Observations from the results: 1.Those with α ∈ (0, 1] focus more on the good condition 2.Those with α ∈ [-1, 0) focus more on the bad condition 3.About 10% of the GMOS are either smaller than the MIN or larger than the MAX 4.From the figure, about 50% α ∈ [-0.2, 0.2], so we set the default value of α to be 0 5.From the figure, the average value of all the 392 α is 0.093. Statistically, these subjects are “optimistic” on average. 6.This value is application dependent.

9. Leader Selection Strategy Several types of topologies supporting multi-party voice conference We will mainly focus on the “end system mixing” topology 1. Simplest topology 2. Bandwidth saving 3. No dedicated server 4. Effective and suitable

10. Leader Selection Strategy For simplicity, we need to make an assumption: The leader’s GMOS L will represent all participants’ opinion on evaluating the overall quality of the whole voice conference. 1. GMOS of each participant 2. GMOS L and GMOS M 3. N GMOS L : (GMOS L1, …, GMOS LN ) 4. Leader with MAX(GMOS Li )

11. Leader Selection Strategy Problem: 1. Asking participants to provide GMOS on the overall QoS is a subjective test. 2. It is impossible to obtain subjective GMOS before or during the voice session. But we need to select the suitable conference leader before the voice session starts. => we need to estimate the GMOS

12. Two-Step Mapping Method W e know how to estimate MOS from measurable network parameters: e.g. packet loss rate and end-to-end delay Loss rate  I e,eff Delay  I d E-Model R-value Estimated MOS R = 93.2 – I e,eff – I d

13. Two-Step Mapping Method Estimated GMOS.

14. The first step of TSMM I e is codec dependent. From the Skype measurement paper, Skype uses iLBC codec as its audio codec. Random loss model Ppl[%] = ppl * 100%, where ppl is packet loss rate

15. The first step of TSMM 2-state Markov loss model: Not use because of rare occurrence BurstR < 1 BurstR = 1 (Random loss) BurstR > 1 (Bursty loss)

16. The first step of TSMM No existing model and we are not certain whether it works well or not. I d could be calculated: Then we can obtain R-value through E-Model: For G.107 codec

17. The second step of TSMM Apply the estimated MOSes derived in the first step and the GMOS model to get estimated GMOS. α= 0.093 is the average value of 392 samples obtained from our experiments α= 0 is the default value.

18. Experiment setup and results The second part of the experiments # of computers LocationAccessing Internet 3ShanghaiADSL 1SJTUCampus LAN 1CUHKCampus LAN SKYPE (Ver 2.5) Ethereal (Ver 0.99) Audition(Ver 1.5)

19. Experiment setup and results The second part of the experiments Through packet trace, we can obtain:  Bit rate,  Jitter,  Loss rate (random loss),  Loss rate and state transfer Prob. (2-state Markov loss model) Organize participants change leader in turn to finish group of experiments.

20. Experiment setup and results Mapping from estimated GMOS to subjective GMOS

21. Experiment setup and results Data results of the second part of the experiments Data with “*” is the largest GMOS value in each group, which means the one it belongs to should be the conference leader.

22. Applications A) Different kinds of metrics like USI. B) Try test data to select the proper leader. C) It could be an indicator. D) Choose a leader candidate if necessary.

23. Conclusion A)Proposal of GMOS B)Method to estimate GMOS C)Leader selection strategy D)Experimental validation

24. Thanks ! Q & A