1. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Emery Berger University of Massachusetts, Amherst Operating Systems CMPSCI 377 Lecture 14: VM Meets the Real World

2. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 2 Last Time: Demand-Paged VM Reading pages Swap space Page eviction Cost of paging Page replacement algorithms Evaluation

3. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 3 Virtual Memory in the Real World Implementing exact LRU Approximating LRU Hardware Support Clock Segmented queue Multiprogramming Global LRU Working Set

4. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 4 Implementing Exact LRU On each reference, time stamp page When we need to evict: select oldest page = least-recently used A, B, C, B, C, C, D

5. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 5 Implementing Exact LRU On each reference, time stamp page When we need to evict: select oldest page = least-recently used A1A1 A, B, C, B, C, C, D

6. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 6 Implementing Exact LRU On each reference, time stamp page When we need to evict: select oldest page = least-recently used A1A1 B2B2 A, B, C, B, C, C, D

7. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 7 Implementing Exact LRU On each reference, time stamp page When we need to evict: select oldest page = least-recently used A1A1 B2B2 C3C3 A, B, C, B, C, C, D

8. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 8 Implementing Exact LRU On each reference, time stamp page When we need to evict: select oldest page = least-recently used A1A1 B4B4 C3C3 A, B, C, B, C, C, D

9. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 9 Implementing Exact LRU On each reference, time stamp page When we need to evict: select oldest page = least-recently used A1A1 B4B4 C5C5 A, B, C, B, C, C, D

10. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 10 Implementing Exact LRU On each reference, time stamp page When we need to evict: select oldest page = least-recently used A1A1 B4B4 C6C6 A, B, C, B, C, C, D

11. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 11 Implementing Exact LRU On each reference, time stamp page When we need to evict: select oldest page = least-recently used A1A1 B4B4 C6C6 A, B, C, B, C, C, D D7D7 LRU page How should we implement this?

12. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 12 Implementing Exact LRU: Data Structures Could keep pages in order – optimizes eviction Priority queue: update = O(log n), eviction = O(log n) Optimize for common case! Common case: hits, not misses Hash table: update = O(1), eviction = O(n)

13. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 13 Cost of Maintaining Exact LRU Hash tables: too expensive On every reference: Compute hash of page address Update time stamp Unfortunately: 10x – 100x more expensive!

14. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 14 Cost of Maintaining Exact LRU Alternative: doubly-linked list Move items to front when referenced LRU items at end of list Still too expensive 4-6 pointer updates per reference Can we do better?

15. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 15 Virtual Memory in the Real World Implementing exact LRU Approximating LRU Hardware Support Clock Segmented queue Multiprogramming Global LRU Working Set

16. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 16 Hardware Support Maintain reference bits for every page On each access, set reference bit to 1 Page replacement algorithm periodically resets reference bits A1A1 B1B1 C1C1 A, B, C, B, C, C, D

17. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 17 Hardware Support Maintain reference bits for every page On each access, set reference bit to 1 Page replacement algorithm periodically resets reference bits A0A0 B0B0 C0C0 A, B, C, B, C, C, D reset reference bits

18. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 18 Hardware Support Maintain reference bits for every page On each access, set reference bit to 1 Page replacement algorithm periodically resets reference bits A0A0 B1B1 C0C0 A, B, C, B, C, C, D

19. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 19 Hardware Support Maintain reference bits for every page On each access, set reference bit to 1 Page replacement algorithm periodically resets reference bits A0A0 B1B1 C1C1 A, B, C, B, C, C, D

20. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 20 Hardware Support Maintain reference bits for every page On each access, set reference bit to 1 Page replacement algorithm periodically resets reference bits A0A0 B1B1 C1C1 A, B, C, B, C, C, D

21. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 21 Hardware Support Maintain reference bits for every page On each access, set reference bit to 1 Page replacement algorithm periodically resets reference bits Evict page with reference bit = 0 Cost per miss = O(n) A0A0 B1B1 C1C1 A, B, C, B, C, C, D D1D1

22. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 22 Virtual Memory in the Real World Implementing exact LRU Approximating LRU Hardware Support Clock Segmented queue Multiprogramming Global LRU Working Set

23. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 23 The Clock Algorithm Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame B1B1 C1C1 A1A1 D1D1 A, B, C, D, B, C, E, F, C, G

24. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 24 The Clock Algorithm B1B1 C1C1 A1A1 D1D1 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame

25. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 25 The Clock Algorithm B1B1 C1C1 A1A1 D1D1 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame

26. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 26 The Clock Algorithm B1B1 C1C1 A0A0 D1D1 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame

27. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 27 The Clock Algorithm B0B0 C1C1 A0A0 D1D1 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame

28. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 28 The Clock Algorithm B0B0 C0C0 A0A0 D1D1 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame

29. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 29 The Clock Algorithm B0B0 C0C0 A0A0 D0D0 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame

30. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 30 The Clock Algorithm B0B0 C0C0 A0A0 D0D0 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame E1E1

31. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 31 The Clock Algorithm B0B0 C0C0 A0A0 D0D0 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame E0E0

32. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 32 The Clock Algorithm B0B0 C0C0 A0A0 D0D0 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame E0E0 F1F1

33. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 33 The Clock Algorithm B0B0 C0C0 A0A0 D0D0 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame E0E0 F1F1 C1C1

34. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 34 The Clock Algorithm B0B0 C0C0 A0A0 D0D0 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame E0E0 F0F0 C1C1

35. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 35 The Clock Algorithm B0B0 C0C0 A0A0 D0D0 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame E0E0 F0F0 C1C1 C0C0

36. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 36 The Clock Algorithm B0B0 C0C0 A0A0 D0D0 A, B, C, D, B, C, E, F, C, G Variant of FIFO & LRU Keep frames in circle On page fault, OS: Checks reference bit of next frame If reference bit = 0, replace page, set bit to 1 If reference bit = 1, set bit to 0, advance pointer to next frame E0E0 F1F1 C1C1 C0C0 G1G1

37. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 37 Enhancing Clock Recall: we don’t write back unmodified pages Idea: favor eviction of unmodified pages Extend hardware to keep another bit: modified bit Total order of tuples: (ref bit, mod bit) (0,0), (0,1), (1,0), (1,1) Evict page from lowest nonempty class

38. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 38 Page Replacement in Enhanced Clock OS scans at most three times Page (0,0) – replace that page Page (0,1) – write out page, clear mod bit Page (1,0), (1,1) – clear reference bit Passes: 1. all pages (0,0) or (0,1) 2. all pages (0,1) - write out pages 3. all pages (0,0) – replace any page Fast, but still coarse approximation of LRU

39. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 39 Segmented Queue Real systems: segment queue into two parts approximate for frequently-referenced pages e.g., first 1/3 page frames – fast exact LRU for infrequently-referenced pages last 2/3 page frames; doubly-linked list – precise How do we move between two segments? clockexact LRU

40. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 40 Virtual Memory in the Real World Implementing exact LRU Approximating LRU Hardware Support Clock Segmented queue Multiprogramming Global LRU Working Set

41. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 41 Multiprogramming & VM Multiple programs compete for main memory Processes move memory from and to disk Pages needed by one process may get squeezed out by another process  thrashing - effective cost of memory access = cost of disk access = really really bad Must balance memory across processes to avoid thrashing

42. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 42 Global LRU Put all pages from all processes in one pool Manage with LRU (Segmented Queue) Used by Linux, BSD, etc. Advantages: Easy Disadvantages: Many

43. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 43 Global LRU Disadvantages No isolation between processes One process touching many pages can force another process’ pages to be evicted Priority ignored, or inverted All processes treated equally Greedy (or wasteful) processes rewarded Programs with poor locality squeeze out those with good locality Result: more page faults

44. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 44 Global LRU Disadvantages, Cont. “Sleepyhead” problem Intermittent, important process Every time it wakes up – no pages! – back to sleep... Susceptible to denial of service Non-paying “guest”, lowest priority, marches over lots of pages – gets all available memory Alternatives?

45. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 45 Working Set Denning: Only run processes whose working set fits in RAM Other processes: deactivate (suspend) Working set = pages touched in last  references Provides isolation A process’s reference behavior only affects itself

46. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 46 Working Set Problems Algorithm relies on key parameter,  How do we set  ? Is there one correct  ? Different processes have different timescales over which they touch pages Not acceptable (or necessarily possible) to suspend processes altogether Not really used Very rough variant used in Windows

47. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 47 Alternative Approach Just buy more RAM! Simplifies memory management Workload fits in RAM = no more swapping! Sounds great…

48. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 48 Memory Prices Over Time “Soon it will be free…”

49. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 49 Memory Prices: Inflection Point!

50. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 50 Memory Is Actually Expensive Desktops: Most ship with 256MB 1GB = 50% more $$ Laptops = 70%, if possible Limited capacity Servers: Buy 4GB, get 1 CPU free! Sun Enterprise 10000: 8GB extra = $150,000! Fast RAM – new technologies Cosmic rays… 8GB Sun RAM = 1 Ferrari Modena

51. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 51 New Approach: SAVMM Scheduler-Aware Virtual Memory Management Scheduler: proportion of desired CPU time Like lottery scheduling VM: allocates memory to meet proportion Inversely related to miss rate Use recent reference behavior Can calculate misses for any size allocation Picks allocation: highest throughput, nearest proportion In development (Berger, Kaplan et al.) Simulation & Linux kernel

52. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 52 Summary: Virtual Memory in the Real World Implementing exact LRU Approximating LRU Hardware Support Clock Segmented queue Multiprogramming Global LRU Working Set

53. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 53 Summary: If You Have to Spend $$... more memory: bad more Ferraris: good