1 Practical, transparent operating system support for superpages Juan Navarro, Sitaram Iyer, Peter Druschel, Alan Cox OSDI 2002

2 What’s a Superpage? A very large page size, much greater than the base page size Supported by most computer architectures today Machines that support superpages usually have several different page sizes, beginning with the base page and then in increasing sizes, each a power of 2 – today, some as large as a gigabyte.

3 Background Summary Virtual memory automates the movement of a process’s address space (code and data) between disk and primary memory. Virtual addresses are translated using information stored in the page table. Page tables are stored in primary memory. Extra memory references due to page table degrades performance

4 Translation Lookaside Buffer TLB (translation lookaside buffer) – faster memory; caches portions of the page table If most memory references “hit” in the TLB, the overhead of address translation is acceptable. TLB coverage: the amount of memory that can be accessed strictly through TLB entries.

5 The Problem Computer memories have increased in size faster than TLBs. TLB coverage as a percentage of total memory has decreased over the years. –At the time this paper was written, most TLBs covered a megabyte or less of physical memory –Many applications have working sets that are not completely covered by the TLB Result: more TLB misses, poorer performance.

6 The Solution Superpages! Increase coverage without increasing TLB size. How? –By increasing amount of memory each TLB entry can map

7 Hardware-Imposed Constraints Must have enough contiguous free memory to store each superpage Superpage addresses (physical and virtual) must be aligned on the superpage size: e.g., a 64KB SP must start at address 0, or 64KB, or 128KB, etc. TLB entry only has one set of bits (R, M, etc.) and thus can only provide coarse-grained info – not good for efficient page management.

8 Design Issues Issues for a superpage management system: –Storage allocation and fragmentation control –Promotion –Demotion –Eviction

9 Issues: Frame Allocation When a page fault occurs, must choose a frame for the new page –In non-superpage systems any frame will do –In a superpage system we may later decide to include this page in a superpage – how does this affect the decision? Possible approaches to allocation: –Reservation based –Relocation based

10 Reservation-based Allocation When a page is initially loaded choose a superpage size and reserve aligned, contiguous frames to hold it. –As other pages are referenced, load them into the previously reserved frames Will adjoining pages ever be needed by the program?

11 Figure 2: Reservation based allocation Allocated framesUnused page frame reservation Virtual address space Physical address space Superpage alignment boundaries Object mappingMapped pages

12 Relocation-based Allocation Wait until a superpage is formed, then move pages to contiguous locations –Incurs overhead of moving pages when superpages are created. Tradeoff: relocation costs versus unused reservations (internal fragmentation)

13 Choosing a Page Size Regardless of whether reservation-based or relocation-based allocation is used, size of superpage must be chosen also. When a computer has several page sizes (base page + several larger sizes), how to choose which size to use? The issue: larger versus smaller

14 Choosing a Page Size Possibilities: –the largest superpage size available –a superpage size that most closely matches the VM object the page belongs to –a smaller size, based on memory availability. Tradeoff: possible performance gains from large page versus possible loss of contiguous physical memory space that may be needed later

15 Large Pages? Large page sizes increase TLB coverage the most, optimize I/O. But … they can also greatly increase the memory requirements of a process –Some pages are only partially filled –Small localities = a kind of internal fragmentation (page only partially referenced) –If pages are not filled or have internal fragmentation, paging traffic can actually increase instead of decrease.

16 Small Pages? Small page sizes reduce internal fragmentation (amount of wasted space in an allocated block & the amount of unreferenced content in a loaded page). But … they have all the problems that large pages solve, plus they also have the possibility of causing more page faults.

17 So Why Not Use Multiple Page Sizes? Memory management is more complex –Uniform page size is simple Multiple page sizes causes external fragmentation –It’s hard to maintain blocks of contiguous free space to accommodate large superpages.

18 SP1 SP3 SP4 SP4 leaves, is replaced by SP5. SP2 leaves. No room for a large superpage External fragmentation SP2 SP5 SP3 SP1

19 Fragmentation Control Memory can become fragmented with reservation-based approach and pages of various sizes. Possible solutions: –Page out or overwrite areas of memory that haven’t been used recently –Preempt unused portions of existing reservations

20 Issues Issues for a superpage management system: –Allocation and fragmentation control –Promotion –Demotion –Eviction

21 Issues: Promotion Initially, base pages are treated normally. Promote when enough pages have been loaded to justify creating a superpage: –Combine TLB entries into one entry –Load remaining pages, if necessary, to fill reservation Promotion may be incremental Tradeoff: early promotion (before all base pages have been faulted in) reduces TLB misses but wastes memory if all pages of the superpage are not needed; late promotion delays benefits of greater TLB coverage.

22 Issues Issues for a superpage management system: –Allocation, fragmentation, promotion – done –Promotion - done –Demotion & eviction

23 Issues: Demotion Reduce superpage size –To individual base pages –To a smaller superpage All or some of the base pages may have been chosen for eviction Difficulty: use bits and dirty bits in the TLB aren’t as helpful as if they referred to a base page. –If the dirty bit is set, the entire superpage must be written to disk, even if only part of it has changed.

24 Design of System Proposed by Navarro, et al. The system discussed in this paper is reservation-based. It supports multiple superpage sizes to reduce internal fragmentation –Effect on external fragmentation? It demotes infrequently referenced pages to reclaim memory frames It is able to maintain contiguity (large blocks of contiguous free frames) without using compaction

25 Design Decisions in This System With respect to allocation and fragmentation –Storage Management –Reservation-based allocation Choosing a page size –Fragmentation control

26 Storage Management Free space (available for reservations) is stored on multiple lists, ordered by superpage size –Buddy system is used for allocation Partially filled reservations are kept on a multi-list (one list for each page size) by largest page size that can be obtained by preempting unused portion Population maps track allocated portions of reservations

27 Frame Allocation A page fault triggers a decision: does the page have an existing reservation or not? If not, then –select a preferred SP size, –locate a set of contiguous, aligned frames –load the page into the correct (aligned) frame –enter the mapping in the page table –reserve the remaining frames Or, load the page into a previously reserved frame & enter mapping in PT

28 Choosing a Superpage Size in the Navarro System Since the decision is made early, can’t decide based on process’s behavior. Base decision on the memory object type; prefer too large to too small –If the decision is too large, it is easy to reclaim the unneeded space –If the decision is too small, relocation is needed

29 Guidelines for Choosing Superpage Size For fixed size memory objects (e.g. code segments) reserve the largest super page possible that is not too large. For dynamic-sized objects (stacks, heaps) that grow one page at a time: allocate extra space for growth.

30 Preempting Reservations in the Navarro System After a page fault, if the guidelines call for a superpage that is too large for any available free block: –Reserve a smaller size superpage or –Preempt an existing reservation that has enough unallocated frames to satisfy the request This system uses preemption wherever possible.

31 Preemption Policy - LRA Which reservation is preempted if more than one can satisfy the request? Choose the one “whose most recent page allocation occurred least recently” - LRA –Reason: spatial locality suggests that related pages will all be accessed fairly closely together in time;(e.g., arrays, memory mapped files). If a reservation hasn’t added new pages recently, it’s unlikely to do so any time soon.

32 Fragmentation Control Contiguity (of storage) is a contended resource Memory becomes fragmented due to –Multiple page sizes –Wired pages (can’t be paged out) Result: not enough large, properly aligned blocks of free memory. Navarro et al. propose several implementation techniques to address this problem

33 Fragmentation Control in the Navarro System* The “buddy allocator” (free list manager) maintains multiple lists of free blocks, ordered by size –When possible, coalesce adjacent blocks of free memory to form larger blocks. Modify the page replacement daemon to include contiguity as one of the factors to be considered.

34 Navarro System: Design Decisions With respect to promotion, demotion & eviction –Incremental promotions –Speculative demotions –Paging out dirty superpages

35 Promotion & Demotion Navarro et. al. implement incremental promotion –e.g., if 4 aligned pages of a 16 page reservation becomes filled, promote to a mid- size superpage Demotion: when a base page is evicted, its superpage is demoted. –Speculative demotion: demote active superpages to determine if the whole page is still in use or just parts

36 Paging Out Dirty Superpages* If a dirty superpage is to be flushed to disk, there is no way to tell if one page is dirty or all pages. Writing out the entire superpage is a huge perfomance hit. Navarro, et. al’s solution: Don’t write to clean superpages. –If a process tries to write to a SP, demote the SP. –Repromote later if all base pages are dirty. They also experimented with a content hash which could tell if a page had been changed

37 Goal of Superpage Management Systems Good TLB coverage with minimal internal fragmentation Navarro, et. al. Conclusion: create the largest superpage possible that isn’t larger than the size of the memory object (except for stack/heap). If there isn’t enough memory, preempt existing reservations (these pages had their chance)

38 Current Usage Superpages were most often used at the time this paper was written to store portions of the kernel and various buffers. –Reason: the memory requirements for these objects are static and can be known in advance. –Superpage size can be chosen to fit the object. More likely to be implemented in clusters and large servers than in desktop machines.

39 Current Research This paper focuses on supporting superpage use in application memory, as opposed to kernel memory. An ongoing research area: memory compaction – whenever there are idle CPU cycles, work to establish large contiguous blocks of free memory –Compare to disk management

40 Summary: Potential Advantages of Superpages Ideally, superpages can improve performance –Without increasing size of TLB (which would be expensive and increase TLB access time) –Without increasing base page size (which can lead to internal fragmentation) Superpages allow use of small (base) and large (super) page sizes at the same time.

41 Summary - Tradeoff Large superpages increase TLB coverage Large superpages are more likely to fragment memory. (Why?) Benefits of large superpages must be weighed against “contiguity restoration techniques” –Pages loaded into reserved areas must be loaded at the proper offset. –Must be enough space for the entire superpage –More overhead for free space management

42 Authors’ Conclusions Can achieve 30%-60% improvement in performance, based on tests using an accepted set of benchmark programs as well as actual applications. Must employ contiguity restoration techniques: demotion, preemption, compaction Must be able to support a variety of page sizes

43 Conclusion Superpage management can be transparently integrated into an existing OS (FreeBSD, in this case). –“hooks” connect the OS to the superpage module at critical events: page faults, page allocation, page replacement, etc. Tests show this technique scales well, according to authors.

44 Follow-up “Supporting superpage allocation without additional hardware support”, Mel Gorman, Patrick Healy, Proceedings of the 7th International Symposium on Memory Management, 2008

45 Premise Fragmentation control is essential for successful implementation of superpages. Navarro’s approach doesn’t always work. Major hindrance: “wired” pages – pages that can’t be paged out or moved – tend to become scattered throughout memory (Navarro addressed this issue; proposed to monitor creation of kernel wired pages, cluster them in one location.)

46 Another problem: page replacement processes that don’t consider superpage structure –Reclaim pages based on age, does not consider contiguity. [Note: Navarro system does claim to take this into consideration – such as activating the paging daemon whenever the system fails to satisfy a request for a certain super-page size]

47 GPBM Grouping Pages By Mobility (GPBM) is a placement policy described by Gorman & Healy that allocates frames to pages based on whether or not the pages can later be relocated. Treats the address space as if divided into arenas, which correspond in size to the largest superpage.

48 Page Mobility Types Movable – no restrictions; can be relocated as long as PT is updated Reclaimable – kernel pages that can be added to the free list (certain kinds of caches, for example) Temporary – pages that are known to be needed for a short time; treated as reclaimable Non-reclaimable – wired pages

49 How are the classes used? Group pages of the same type into arenas of the same type. The number of movable and reclaimable arenas have the most effect on the number of superpages that can be allocated. Contiguity-aware page replacement is used.

Summary Superpages promise performance improvement but so far no generally accepted approach for user level pages. Reservation based approach seems to be most popular Contiguity is the biggest problem Some researchers propose hardware solutions, such as re-designing the memory controller to allow holes in SPs, or re-designing TLB to permit SPs that consist of non-contiguous base pages. –To date, no hardware solutions implemented. 50