1. 1 Ashish Gupta Bin Lin Peter Dinda Department of Computer Science Northwestern University

2. 2 Overview Our Goal –Introduce resource borrowing and its motivation –Current model and further possibilities –What we want to study ? Methodology –Actually try it out ! –Design of a system which emulates resource borrowing and captures feedback Design of our application –What it does ? –Extensive System Monitoring –Resource Exercisers we have For technical details refer to paper and tech report System Design – the client server system, creating and analyzing testcases The controlled study –What the user was asked to do, user details etc., set of testcases ( ramp and step) Results –Aggregated –Context –Person –Frog in the pot ? –Hardware ? Internet Wide Study Conclusions and Results

3. 3 Outline Motivation behind Resource Borrowing Measuring user comfort System Design for Measuring User Comfort Controlled Study Its Results Conclusions

4. 4 New Models of Distributed Computing Break Screen Saver time Working 9 am 11 am 1 pm3 pm4 pm5 pm 9 am

5. 5 Observation and ideas Idea: Why not borrow the unused resources ? Unused Resources Problem: Performance Slowdown

6. 6 Effect of performance slowdown

7. 7

8. 8

9. 9 The question we want to answer Depends on Other Factors as well… What level of resource borrowing leads to user discomfort for significant fraction of users ?

10. 10 CPURAM Disk

11. 11 ContextPersonHardware Others ?

12. 12 Power UsersTypical UsersBeginners

13. 13

14. 14 Outline Motivation behind Resource Borrowing Measuring user comfort System Design for Measuring User Comfort Controlled Study Its Results Conclusions

15. 15 How can we find this out ? Actually test it with user feedback A system for emulating resource borrowing and measuring user discomfort using direct feedback http://comfort.cs.northwestern.edu

16. 16 How can we find this out ? 1 Measure existing contention

17. 17 How can we find this out ? Emulate Resource Borrowing Real Contention Borrowed Contention Final Contention 2 % CPU Usage vs. Time

18. 18 How can we find this out ? Allow User Feedback 3 F11

19. 19 How can we find this out ? Find MAX Level of tolerable contention Borrowed Contention Final Contention 4 MAX % CPU Usage vs. Time

20. 20 How is it useful ? Comfort = f (Contention) To find a quantitative relationship between resource borrowing and user comfort

21. 21 How is it useful ? Enhances the degree of workstation sharing

22. 22 Interactivity Aware Scheduling Interactive More intelligent scheduling of interactive tasks possible via user feedback and our results

23. 23 Virtual Machine Scheduling Virtuoso: Distributed Computing with Virtual Machines User pays for the resources he uses Need to know minimal resources needed to keep an interactive VM user happy

24. 24 Outline Motivation behind Resource Borrowing Measuring user comfort System Design for Measuring User Comfort Controlled Study Its Results Conclusions

25. 25 Testcase Transfer Results Transfer Server Client Resource borrowing using testcase profile Can do large scale user studies and maintain testcase control over wide-area

26. 26 The client design

27. 27 Testcases Each testcase stores details of resource borrowing Testcase consists of exercise functions for each resource Exercise function is a vector of contention values Example: CPU contention: [0,0.5,1.0,1.5,2.0] at 1 Hz

28. 28 Exercise Functions Time Contention Exercise Function 1 Exercise Function 2 Exercise Function 3 Exercise Function 4 Has all sort of strange exercise functions Power-law, exp, pareto Plot our real testcases

29. 29 Flowchart of Testcase Execution No Testcase running Resource Borrowing Phase Begins (Testcase Runs) No user feedback User expressed discomfort Testcase Exhausts after some time Testcase Ends immediately User feedback

30. 30 Resource Exercisers CPU Exerciser –Contention describes the expected extra number of threads in ready queue –Fractional resource borrowing using stochastic scheduling methods –Validated to contention level of 10 Disk Exerciser –Random seek and read/write in a large file (twice the memory) –Validated to contention level of 7 Memory Exerciser –Borrows a fraction of physical memory: f rom 0 to 1

31. 31 Memory Exerciser Contention indicates fraction of total memory allocated in addition to occupied memory Pages are touched frequently to keep them in physical memory 0MB 1024 MB Occupied Mem. 0.5 Memory Contention Clipping Point

32. 32

33. 33 Outline Motivation behind Resource Borrowing Measuring user comfort System Design for Measuring User Comfort Controlled Study Its Results Conclusions

34. 34 The control study description 33 undergrad and grad students –Note: may not be rep. of PC users population 1.5 hrs each Each user was assigned 4 popular tasks to do –MS Word –MS Powerpoint –MS Explorer  searching and saving information –Quake III

35. 35 Machine Configuration

36. 36 Testcases for the Control Study Three types of testcases: step, ramp and blank Ramp allows us to know the point of discomfort for each resource Step and Ramps: comparing the manner of borrowing resources Blank: To test the background level of discomfort

37. 37 Frog in the Pot Hypothesis % CPU Contention vs. Time Step/Ramp Dynamics

38. 38 Testcases for the Control Study Testcases different for each application –Some apps more intensive than others Example of Word and Quake Requires Calibration

39. 39 Outline Motivation behind Resource Borrowing Measuring user comfort System Design for Measuring User Comfort Controlled Study Its Results Conclusions

40. 40 Resource borrowing vs User Discomfort CDFs for discomfort contention level –Not all contentions cause discomfort: exhausted region CDFs allow us to read %age of people discomforted for a given contention –We want to borrow resources such that only a small fraction are discomforted –Metric c 0.05 : At what contention do we discomfort only 5% of the people ?

41. 41 C 0.05 = 0.35 ( aggregated over all applications) Exhausted Region Discomfort Region Aggregated Results

42. 42 Resource borrowing vs User Discomfort CDFs for discomfort contention level –Not all contentions cause discomfort: exhausted region CDFs allow us to read %age of people discomforted for a given contention –We want to borrow resources such that only a small fraction are discomforted –Metric c 0.05 : At what contention do we discomfort only 5% of the people ?

43. 43 Results overview Aggregated Results Dependence on Resource Dependence on Context Dependence on Users Dependence on Step/Ramp

44. 44 Aggregated Results

45. 45 C 0.05 = 0.35 ( aggregated over all applications) Exhausted Region Discomfort Region Aggregated Results

46. 46 Can borrow a lot, discomforting only a small fraction\ C0.05=0.33 Aggregated Results Exhausted Region Discomfort Region

47. 47 Can borrow a lot, discomforting only a small fraction C 0.05 = 1.11 Aggregated Results Exhausted Region Discomfort Region

48. 48 Dependence on Resource

49. 49 An overview of factors Dependence on Resource User sensitivity by task and resource (Low, Medium, High) Users more tolerant for disk and memory borrowing, than CPU Results Qualitative, but the testcase range is wide

50. 50 CPU Dependence on Resource

51. 51 Memory Dependence on Resource

52. 52 Disk Dependence on Resource

53. 53 Dependence on Context

54. 54 Comparison of CPU discomfort Word IE Powerpoint Quake Dependence on Context

55. 55 Dependence On Context - CPU Word Powerpoint IE Quake Dependence on Context

56. 56 Dependence on Context - Memory Word Powerpoint IE Quake Dependence on Context

57. 57 Dependence on Context - Disk Word Powerpoint IE Quake Dependence on Context

58. 58 Fraction of users discomforted in Disk Word Powerpoint IE Quake Dependence on Context

59. 59 Dependence on User

60. 60 Dependence on User Power Typical Beginner Dependence on User

61. 61 Step/Ramp Dynamics

62. 62 Frog in the Pot Hypothesis % CPU Contention vs. Time Step/Ramp Dynamics

63. 63 An example with PPT % CPU Contention vs. Time 0.98 Step/Ramp Dynamics

64. 64 Internet Wide-Area Study To gain more insight into our results No controls here, hope to get more general and complete data Users can download a client from http://comfort.cs.northwestern.edu http://comfort.cs.northwestern.edu Clean installation/un-installation and privacy options Join us !

65. 65 Outline Motivation behind Resource Borrowing Measuring user comfort System Design for Measuring User Comfort Controlled Study Its Results Conclusions

66. 66 Conclusion Disk and memory can be borrowed more aggressively, CPU less so –Currently Seti@home, P2P file sharing are very conservative in disk and memory usageSeti@home A throttle for resource borrowing  very useful –Don’t run full blast. Have the ability to carefully control how much you borrow through stochastic scheduling CDFs can be used to set the throttle Knowing the user context helps Using Direct user feedback can be useful –We are moving ahead in this direction, esp in scheduling of Virtual Machines for satisfactory interactive performance

67. 67 Conclusion Comfort = f(contention) Disk and Memory surprisingly easy to borrow, CPU borrowing also liberal Set of empirical CDFs provided Depends of other factors: Resource, Context and Users

68. 68 Current and Future Work User feedback for interactive VM Scheduling Real time Scheduling to give exact resources to VMs Minimal resources required to keep the user happy

69. 69 Acknowledgements http://comfort.cs.northwestern.edu Special Thanks to: Don Norman Andrew Ortony Ben Watson Jack Tumblin Leslie Lamport and All our beta testers Study participants

70. 70 The placebo effect Users express discomfort even when no testcase is running Can never guarantee that the user will never express feedback

71. 71 Related Work

72. 72 Questions and Answers Other types of irritation Bursty natures of Desktop Computing, bursts indicate sudden action-response tasks Large number of factors for irritation Since there is usually a power law for process run times, why do we need this model ? (CPU is idle most of the time) Why can’t we just run the bg app at a lower priority ? It may benefit much more than just running at a fixed contention. Memory exerciser: What does contention mean ? How do you find out free physical memory ? How do you avoid thrashing ? Why do you avoid thrashing ? How does the CPU exerciser work ? Why do you disable buffering in the Disk exerciser ? The aggregated graphs are not useful. Memory usage may be much more dynamic then in your tests due to user multitasking etc. So this may not represent reality ? –Our study is a start in this direction and we realize that there are limitations to the data we have collected. But we expect to collect and analyze more results from the Internet Wide area study and generalize our results to more real environments. There are no controls in the internet wide area study. The control study was meant to specifically answer the questions we have posed, like the affect of other factors in user discomfort and resource borrowing.

73. 73 Lower priority ? –Disk and memory ? –There is a limit (max) possible amount of cycles stolen. Our goal is to s=estimate that. Seperately, there are many possible sched mech that would get closer to that max. Lower priority is an example. Linger longer is an example. Screen saver is an example. –Quake example –Don’t argue for/against a particular scheduling scheme –It is a step in the right direction… (for results)

74. 74 Testcase Transfer Results Transfer Server Client Resource borrowing using testcase profile

75. 75 The client-server system Data Exchange Hotsync –Results transfer –Testcase update Testcase Tools

76. 76

77. 77

78. 78 An irritating Interface

79. 79 Interactive Computation Bursty Nature of Interactive Computation

80. 80

81. 81

82. 82 Understanding Testcases CPU ramp 1 ramp 2 ramp 3 step 1 step 2 ….. MEM ramp 1 ramp 2 ramp 3 step 1 step 2 ….. Ramp Step

83. 83 Combination of Resources CPU Usage Memory Usage Hard Disk activity Network Traffic Comfort Zone surface

84. 84 % CPU Contention vs. Time

85. 85