1. 3.1 Advanced Operating Systems Superpages TLB coverage is the amount of memory mapped by TLB. I.e. the amount of memory that can be accessed without TLB misses. Low TLB coverage can cause many unnecessary memory references. Large and complex TLBs make it difficult to attain short processor cycle times.

2. 3.2 Advanced Operating Systems TLB coverage trend TLB coverage as percentage of main memory TLB miss overhead:  5% 5-10%  30% Factor of 1000 decrease in 15 years

3. 3.3 Advanced Operating Systems Superpages Keywords Many processors achieve wider TLB coverage by supporting large pages in the TLB. These pages are called Superpages. The smaller pages using by the page table are called base pages. Promotion is the action of upgrading a set of base pages to a superpage in the TLB. Demotion is the opposite action of promotion.

4. 3.4 Advanced Operating Systems Fragmentation Larger internal fragmentation. –Some base pages within the superpage can be unused, while the operating system decides to swap out another page. External fragmentation also can be. –Suppose superpage is N base pages and there are N free base frames, but they are not contiguous.

5. 3.5 Advanced Operating Systems More Problems A thrashing can occur even when there are enough pages. The reading is less efficient. More data is read, but no guarantee that the data is going to be useful. –One dirty bit for a superpage can cause more I/O traffic too. The coarse granularity is not just of the dirty bit, but also of protection attributes (read, write, execute) and reference bit.

6. 3.6 Advanced Operating Systems Multiple Page Size The "Pentium"s support just two sizes of pages – 4KB base pages and 4MB superpages. However, The "Alpha"s support 8KB base pages while the superpages are of 64KB, 512KB or 4MB. The "Itanium"s support 10 different page sizes from 4KB to 256MB. –4KB,8KB,16KB,64KB,256KB,1MB,4MB,16MB,64MB,256MB

7. 3.7 Advanced Operating Systems Superpage Allocation virtual memory physical memory superpage boundaries B B A A C C D D virtual memory physical memory superpage boundaries BACD ABCD

8. 3.8 Advanced Operating Systems Promotion Promotion is gathering some base pages into a superpages, or gathering some small superpages into a bigger superpage. When to promote? –Wait for the process to touch the base pages? May lose opportunity to increase TLB coverage. –Create a small superpage? May waste overhead. –Forcibly populate the base pages? May cause internal fragmentation.

9. 3.9 Advanced Operating Systems Talluri's Superpages on 1994 Talluri and Hill suggested that the OS will try to allocate a base frame that is part of an available, contiguous range of base frames equal in size and alignment to the maximal desired superpage size. The promotion will be done, when the number of base frames in use reaches a promotion threshold. Under memory pressure the reserved area can be preempted.

10. 3.10 Advanced Operating Systems Romer's Superpages On 1995 Romer et al. suggested to allocate base pages everywhere and on promotion to relocate them. –If a base page is move into an occupied frame, the old base page will be swapped out. If the old base page is non-pageable, another relocation is done. Due to the high price of promotions, the OS promotes pages just after a high number of TLB misses when the OS is convinced that the promotion can benefit the performance. The OS is more robust to fragmentation, but the performance is worse.

11. 3.11 Advanced Operating Systems HP-UX and IRIX HP-UX is a Unix version on HP and IRIX is a Unix version on Silicon Graphics. The superpaging support in both of them were published on 1998. In both of these OSes, when a base page is faulted in, the system allocates several contiguous base frames to fault in surrounding base pages and immediately promote them into the maximal desired superpage. This approach is equivalents to Romer's superpages with a promotion threshold of one base page.

12. 3.12 Advanced Operating Systems HP-UX and IRIX's superpage size The preferred superpage size is based on memory availability at allocation time and on a user-specified per-segment page size hint. –This hint is associated with an application binary's text and data segments (added by the compiler). –Size specified by the user is non-transparent and it is against Software Engineering rules. Both HP-UX and IRIX manage the "holes" in the memory by LRU lists for each of the superpage's sizes. –Any "hole" can be just in one LRU list.

13. 3.13 Advanced Operating Systems More on IRIX IRIX has a coalescing daemon that relocates pages when it is possible, so a promotion can be done faster. IRIX is actually a hybrid approach of the reservation-based allocation and the relocation based allocation.

14. 3.14 Advanced Operating Systems Hardware Support On 2001 Fang et al. suggested to solve the reallocation problem by a dedicated hardware. They introduced an additional level of address translation in the memory controller, so that the OS can promote non-adjacent physical pages into a superpage. Unfortunately, this new method is still not adapted by the CPU producers.

15. 3.15 Advanced Operating Systems Demotion Convert a superpage into smaller pages. –Smaller pages can be smaller superpages or base pages. Traditionally demotion has been done under memory pressure when part of a superpage has been swapped out of the memory or when there is no room in the TLB.

16. 3.16 Advanced Operating Systems Implicit Demotion On 2002 Navarro et al. suggested to demote clean superpages whenever a process attempts to write into them, and re-promote (incrementally) later if the other base pages are dirtied. His implementation was on FreeBSD running on an Alpha processor. In the original FreeBSD when a dirty superpage needs to be written to the disk, it will consider all the constituent base pages dirty. Demotion is also done whenever the protection attributes are changed on part of a superpage.

17. 3.17 Advanced Operating Systems Wired Pages Scatter wired (non-pageable) base pages can be a bone in the throat. On 2002 Navarro suggested to reserve an area in the memory for wired allocation. –As a result the wired pages will be clustered.

18. 3.18 Advanced Operating Systems Linux Traditionally Linux does not support superpages. On 2002 Winwood suggested a Linux version supporting superpages. The major deficiency of Linux is that it mainly runs on Pentium machines which have just two sizes of pages – 4KB and 4MB; hence undeveloped.