1. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 1 Reinder J. Bril A QoS approach for Multimedia Consumer Terminals - A case for Conditionally Guaranteed Budgets - 23-11-2004

2. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 2 Multimedia Consumer Terminals and QoS Multimedia Consumer Terminals –audio/video: perception is key –high volume electronics: cost-constrained –requires average-case resource allocation High quality audio and video: –have real-time requirements Quality of Service (QoS) –“collective effort of service performances that determine the degree of satisfaction of the user of that service” (International Telecommunications Union)

3. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 3 Quality of Service Resource Management (QoS-RM) Scalable Video Algorithms (SVA) V-QoS University of Madrid (dit/UPM) University of Illinois at Urbana- Champaign (UIUC) University of Mannheim University of St. Petersburg ITEA/Europa, ITEA/Robocop, OZONE, … Multi-disciplinary QoS approach

4. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 4 Overview Multimedia Consumer Terminals A QoS approach Conditionally Guaranteed Budgets Conclusion

5. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 5 Overview Multimedia Consumer Terminals –Media processing from dedicated HW to SW –Platforms are resource constraint –High quality video has real time requirements A QoS approach Conditionally Guaranteed Budgets Conclusion

6. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 6 Overview Multimedia Consumer Terminals A QoS approach –Adaptive applications –Budget-based resource manager –Control hierarchy Conditionally Guaranteed Budgets Conclusion

7. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 7 Overview Multimedia Consumer Terminals A QoS approach Conditionally Guaranteed Budgets –Resource allocation conflict –Extension of QoS approach –Analysis Conclusion

8. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 8 Multimedia Consumer Terminals DVD CDx front end YC interface IEEE 1394 interface DVB Tuner Cable modem CVBS interface VGA RF Tuner Focus : Receivers in broad-cast environments High-quality video applications

9. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 9 Traditional High-End TV Architecture Traditional TV sets and Set-Top Boxes: Fixed algorithms for fixed HW architectures Upgrade for new services and applications is problematic Systems are not flexible

10. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 10 Digital video platform Expectations: Upgradeable for new services and applications Fast time-to-market for new features Enabling approach for product families

11. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 11 HW Architectures vs. SW Applications Low- end Mid- range High- end Resources Product Families SW-Modules Algorithm 1 Algorithm 2 Algorithm 4 Algorithm 3 Algorithm 1 min max Algorithm 3 Algorithm 2 Algorithm 4

12. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 12 Low- end Algorithm 1 Algorithm 2 Algorithm 4 Algorithm 3 Algorithm 1 Mode 1 Mode 2Mode 3 Flexibility

13. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 13 Low- end Mode 1 Mode 2 Algorithm 3 Alg. 4 Algorithm 1 Alg. 1 Algorithm 3 Scalability

14. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 14 Low- end Initial TransitoryFinal Algorithm 3 Algorithm 1 Alg. 1 Algorithm 3 Alg. 4 Alg. 1 Algorithm 3 Smooth mode transition

15. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 15 Platform constraints Cost Functionality targetlimit traditional systems scalable approach Functionality Quality targetlimit traditional systems scalable approach

16. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 16 Platform constraints Cost-effectiveness requirement –High volume: low bill of material –Low power –Software solutions are relatively expensive (mm 2 silicon / power) –Average case and worst-case are far apart

17. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 17 Fluctuating resource requirements time load structural load running average temporal load worst-case

18. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 18 Fluctuating resource requirements Temporal load changes (very frequent) –from I frame to B frame –more or less motion –transient high peaks Structural load changes (less frequent) –scene change –from movie to camera –from news to commercial

19. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 19 Platform constraints Cost-effectiveness requirement –High volume: low bill of material –Low power –Software solutions are relatively expensive (mm 2 silicon / power) –Average case and worst-case are far apart Conclusion: Aim for average-case resource allocation

20. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 20 High quality video original up-scaled Rendered stream: 60 Hz (TV screen) Input stream: 24 Hz (movie) TV companies invest heavily in video enhancement, e.g. temporal up-scaling

21. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 21 High quality video original up-scaled Input stream: 24 Hz (movie) TV companies invest heavily in video enhancement, e.g. temporal up-scaling displayed Deadline miss leads to “wrong” picture. Deadline misses tend to come in bursts (heavy load). Valuable work may be lost.

22. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 22 High quality video original up-scaled Input stream: 24 Hz (movie) QoS trade-off (at run-time): Lesser picture quality often better than temporal incorrectness. TV companies invest heavily in video enhancement, e.g. temporal up-scaling displayed

23. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 23 High quality video QoS = degree of user satisfaction User satisfaction has to do with perception: –Lesser picture quality often better than temporal incorrectness. –Quality fluctuations are perceived as non-quality. –With a scene change, the brain needs some time to re-focus. –Most people focus on one thing at a time (user focus) –User focus normally is at the center of a window, the window that received the latest (remote) command. Only video specialists can make the necessary trade-offs.

24. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 24 Multimedia Consumer Terminals - Summary Media processing from dedicated HW to SW –Flexibility & scalability –Fast time-to-market –Product families Platform constraints –Aim for average-case resource allocation High quality video –Has real-time requirements; –is about perception -> QoS; –real-time is a QoS parameter; –QoS is primarily an application-domain issue.

25. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 25 Overview Multimedia Consumer Terminals A QoS approach –Adaptive applications –Budget-based resource manager –Control hierarchy Conditionally Guaranteed Budgets Conclusion

26. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 26 User applications Media processing pip mode 2: main + pip main mode 1: main disk mode 3: main + pip + disk Modes InputOutput DVD In mode 3, main + pip + disk can not run at highest quality.

27. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 27 disk : non-scalable main: scalable mixer : non-scalabledigitizer: non-scalablepip: scalable hierarchical task digit Application execution model (prototype) DVD dmux audio dec. sharp enh. mixer audio rend read scaler enc. hw scaler enc. writer dec enc. scalable task task connection to HW IO buffer data transfer Both main and pip are scalable.

28. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 28 Scalable Algorithm Quality Resource needs

29. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 29 s1s1 s2s2 Detail Filter Noise Measurement Amplitude Control FilterACNM No 1DNo 2DNo 2DYes QL0 QL1 QL2 QL3 Example: Sharpness Enhancement

30. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 30 QL0QL1QL4 QL0QL2QL4 Resource range for SVA examples

31. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 31 Budget-based resource manager CPU time app. 1 time budget 1 CPU time app. 2 budget 2 Temporal isolation: –Reserved, guaranteed, and enforced budgets e.g. 20% of the CPU time every 20 ms; –Applications run on a “virtual platform”;

32. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 32 Dynamic load (reminder) Temporal load changes (very frequent) –from I frame to B frame –more or less motion –transient high peaks Structural load changes (less frequent) –scene change –from movie to camera –from news to commercial Mode change (infrequent) –functional change –external trigger

33. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 33 Control hierarchy Dynamic load at different time-scales: –Temporal load changes; –Structural load changes; –Mode change. Layers of control, e.g. –local quality control of applications; –global system utility control.

34. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 34 Adaptive applications Provide quality levels + estimated resource req. Co-operative QoS approach Resource manager Provides guaranteed resource budgets Local quality control SVAs … Global system utility control Optimizes system utility, sets quality levels + allocates resources

35. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 35 Overview Multimedia Consumer Terminals A QoS approach Conditionally Guaranteed Budgets –Why: Resource allocation conflict –How: Extension of QoS approach –Analysis Conclusion

36. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 36 Resource requirements (reminder) time load structural load running average temporal load

37. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 37 time load Resource allocation: worst-case Not cost-effective structural load running average temporal load

38. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 38 time load Resource allocation: close to average Instantaneous budget increase Not yet feasible structural load running average temporal load

39. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 39 time load Resource allocation: close to average Instable output quality Not acceptable for important applications structural load running average temporal load

40. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 40 time load Resource allocation: close to average “wasted” Not cost-effective structural load running average temporal load

41. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 41 Resource allocation conflict Structural load increase + close-to-average resource allocation yields –either instable output quality  not acceptable for important applications –or “wasted” resources  not cost-effective

42. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 42 time load Conditionally guaranteed budgets: Why? Instantaneous budget increase B structural load running average temporal load BB Anticipated increase

43. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 43 Conditionally guaranteed budgets: How? Basic approach (MIA and LIA): –Two modes of quality settings + allocation: Q MIA, B MIA +  B MIA Q MIA, B MIA Q LIA,N, B LIA +  B LIA Q LIA,A, B LIA Anticipated mode (high load for MIA) Normal mode (low load for MIA) MIA LIA CGB

44. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 44 Conditionally guaranteed budgets: How? Adaptive applications Resource manager (RM) Global system utility control MIA LIA Inform MIA, LIA, and RM about both modes Normal mode Anticipated mode Normal mode Anticipated mode Normal mode Anticipated mode modes

45. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 45 Conditionally guaranteed budgets: How? Adaptive applications Resource manager (RM) Global system utility control MIA Normal mode Anticipated mode Normal mode Anticipated mode LIA Normal mode Anticipated mode Running in normal mode

46. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 46 Conditionally guaranteed budgets: How? MIA detects load increase, claims  B MIA, and switches mode Adaptive applications Resource manager (RM) Global system utility control MIA Normal mode Anticipated mode Normal mode Anticipated mode LIA Normal mode Anticipated mode Claim  B MIA Mode transition

47. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 47 Adaptive applications Conditionally guaranteed budgets: How? Resource manager (RM) Global system utility control MIA Normal mode Anticipated mode Normal mode Anticipated mode LIA Normal mode Anticipated mode RM switches mode instantaneously, and informs LIA Inform LIA Mode transition

48. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 48 Adaptive applications Conditionally guaranteed budgets: How? Resource manager (RM) Global system utility control MIA Normal mode Anticipated mode Normal mode Anticipated mode LIA Normal mode Anticipated mode LIA switches mode Running in anticipated mode

49. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 49 Conditionally guaranteed budgets: How? Summary basic approach: –Assumption: Anticipation of resource needs and modes; –Allocation phase: Informing MIA, LIA, and RM about modes; Delegation of mode changes to MIA; –Execution phase: Release and claim of resources by MIA; Instantaneous mode change by RM.

50. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 50 Conditionally guaranteed budgets: How? How to change budgets instantaneously ? In-the-place-of resource consumption –LIA consumes  B LIA exactly when MIA would have consumed  B MIA

51. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 51 In-the-place-of budget consumption B MIA  B MIA MIA LIA Anticipated mode

52. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 52 In-the-place-of budget consumption B MIA  B MIA MIA LIA Normal mode

53. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 53 In-the-place-of budget consumption B MIA  B MIA MIA LIA Normal mode

54. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 54 In-the-place-of budget consumption B MIA  B MIA MIA LIA Claim  B MIA Mode switch

55. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 55 Conditionally guaranteed budgets Analysis How to determine  B LIA ? –Worst-case (i.e. minimal) amount that can be guaranteed on a periodic basis. Cognac-glass algorithm

56. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 56 Cognac-glass algorithm How to determine the worst-case  B LIA ? MIA LIA B MIA  B MIA RR T MIA T LIA

57. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 57 How to determine the worst-case  B LIA ? Worst-case for  R when  B MIA is available: –as early as possible for first overlapping interval; “best-case” analysis –as late as possible for last overlapping interval. “worst-case” analysis –based on notion of advancement Minimize for all values of  R. Cognac-glass algorithm

58. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 58 Cognac-glass algorithm Example –fixed-priority preemptive scheduling –set of four applications ? MIA LIA A2A2 A1A1 29 31 14 6 9 - 2 1 7 periodbudget BiBi BiBi TiTi

59. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 59 Advancement B MIA B MIA +  B MIA MIA A MIA (t)

60. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 60 Advancement B MIA B MIA +  B MIA WA MIA (t) Worst-case advancement

61. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 61 Advancement B MIA B MIA +  B MIA Worst-case and best-case advancement WA MIA (t) BA MIA (t)

62. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 62 Situation of worst-case availability T MIA B MIA B MIA +  B MIA BA MIA (t) WA MIA (t  T MIA )

63. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 63 Situation of worst-case availability T LIA B MIA B MIA +  B MIA RR  B B MIA (  R )  B W MIA (  R +T LIA  T MIA )

64. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 64 Situation of worst-case availability T LIA  B MIA RR  B B MIA (  R )  B W MIA (  R +T LIA  T MIA )  B LIA = RR min (  B B MIA (  R ) +  B W MIA (  R + T LIA – T MIA ))

65. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 65 Situation of worst-case availability T LIA  B MIA RR  B B MIA (  R )  B W MIA (  R +T LIA - T MIA )  B LIA = RR min (  B B MIA (  R ) +  B W MIA (  R + T LIA – T MIA ))

66. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 66 Situation of worst-case availability T LIA  B MIA RR The curve looks like the shape of a glass. Changing the relative phasing  R is like tilting the glass… hence, cognac-glass algorithm.

67. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 67 Situation of worst-case availability T LIA  B MIA  R,lwb Range for  R

68. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 68 Situation of worst-case availability T LIA  B MIA  R,upb Range for  R : [  R,lwb,  R,upb ]

69. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 69 Situation of worst-case availability T LIA  B MIA RR Range for  R Domination values of  R : [  R,lwb,  R,upb ]

70. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 70 Situation of worst-case availability T LIA  B MIA RR Range for  R Domination values of  R : [  R,lwb,  R,upb ]

71. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 71 Situation of worst-case availability T LIA  B MIA RR Range for  R Domination values of  R : [  R,lwb,  R,upb ]

72. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 72 Situation of worst-case availability T LIA  B MIA RR Range for  R Domination values of  R : [  R,lwb,  R,upb ]

73. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 73 Situation of worst-case availability T LIA  B MIA  R,min = 17 Range for  R Domination values of  R : [  R,lwb,  R,upb ]  B LIA  B LIA = 5

74. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 74 Summary of the analysis: –Based on notion of advancement; –requires best-case next to worst-case analysis; –restricted to subset of values for  R. See [Bril 04] for: –a generalization to arbitrary periods T MIA and T LIA ; –formalization; –efficient calculation. Cognac-glass algorithm

75. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 75 Overview Multimedia Consumer Terminals A QoS approach Conditionally Guaranteed Budgets Conclusion

76. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 76 Conclusion [Bril 04] R.J. Bril, Real-time scheduling for media processing using conditionally guaranteed budgets, PhD thesis TU/e, IPA Dissertation Series 2004 – 13, Sept. 2004, http://alexandria.tue.nl.extra2/200412419.pdf.

77. Reinder J. Bril, r.j.bril@tue.nl TU/e Informatica, System Architecture and Networking 77 Acknowledgement Research (PhD): –Emile H.L. Aarts (TU/e); –Gerhard Fohler (Mälardalen University, Sweden); –Wim F.J. Verhaegh (Philips Research); –Christian Hentschel (Brandenburger University, Germany); –Johan J. Lukkien (TU/e); –Peter D.V. v.d. Stok (TU/e). V-QoS program: –All program members and partners; –Clara M. Otero Pérez; –Clemens C. Wűst.