UQC152H3 Advanced OS Memory Management under Linux

Memory Addressing Segmentation and paging background Segmentation –Intel segmentation hardware –Linux use of Intel segmentation Paging –Intel paging hardware –Linux use of Intel paging Memory layout –Kernel layout in physical memory –Process virtual memory layout –Kernel virtual memory layout

Segmentation Background Motivated by logical use of memory areas –Code, heap, stack, etc. Base + offset –Segment registers contain base Segments variable size (usually large) Process as collection of segments –No notion of linear, contiguous memory –Similar to “multi-stream” files (Mac) Requires “segment table”

Paging Background Motivated by notion of linear, contiguous, "virtual" memory (space) –Every process has it's own "zero" address Uniform sized chunks (pages – 4K on Intel) Virtually contiguous pages may be physically scattered Virtual space may have "holes" Page table translates virtual "pages" to physical "page frames"

Intel Segmentation/Paging Intel address terminology: –Logical – segment + offset –Linear (virtual) – 0.. 4GB (64GB with PAE) –Physical Logical -> Linear -> Physical Paging can be disabled Segmentation required –Though you can just have one big segment

Intel Segmentation Hardware Segment registers: cs, ss, ds, es, fs, gs –Indices ("selectors") into "segment descriptor tables" Segment descriptor tables: GDT, LDT –Global and local (per process, in theory) –Each holds about 8000 descriptors Segment descriptors: 8 bytes each –Base/limit, code/data privileges, type –Cached in ro registers when seg registers loaded Task segment descriptor (TSS) –Special segment for holding process context

Segmentation in Linux History –Early versions segmented; now paged –Using "shared" segments simplifies things –Some RISC chips don't support segmentation Linux only uses GDT –LDTs allocated for Windows emulation –Each process has TSS and LDT descriptors –TSS is per-process; LDT is shared

Linux Descriptor Allocation GDT holds 8192 descriptors –4 primary, 4 APM, 4 unused –8180 left / 2 -> 4090 processes (limit in 2.2) Primary (shared) segments –Kernel code, kernel data –User code, user data –Segments overlap in linear address space 2.4 removes 4K process restriction

Intel Paging Hardware Page table "maps" linear address space –Some pages may be invalid –Address space grows/shrinks (mapping) New regions mapped by DLLs, mmap(), brk() –Pages (linear) vs. page frames (physical) Page may map to different frame after swap –Page tables stored in kernel data segment Intel page size: 4K or 4M ("jumbo pages") –Jumbo pages reduce table size Paging "enabled" by writing cr0 register

Intel Two-Level Paging "Page table" is actually a two-level tree –Page Directory (root) –Page Tables (children) –Pages (grandchildren) Linear address carved into three pieces –Directory (10), Table (10), Offset (12) –Entries: frame # + bookkeeping bits Bookkeeping bits: –Present, Accessed, Dirty –Read/Write, User/Supervisor, Page Size Jumbo pages –Map entire 4GB address space with just top-level Directory –No need for Tables! (Kernel uses this technique)

Three-Level Paging How big are page tables on 64 bit arch? –Sparc, Alpha, Itanium –Assume 16K pages => 32 M per process! Better idea: three-level paging trees –Page Global Directory (pgd) –Page Middle Directory (pmd) –Page Table (pt) Carve linear address into 4 pieces Conceptual paging model –On Intel, page middle directory is compiled out

Aside: Caching Exploit temporal and spatial locality L1 and L2 caches (on chip) Cache "lines" (32 bytes on Intel) Kernel developer goals: –Keep frequently used struct fields in the same cache line! –Spread lines out for large data structures to keep cache utilized –The "Cache Police"

TLB Translation Look-aside Buffer –Virtual to physical cache Must be flushed (invalidated) when address space mappings change A significant cost of context switch

Paging in Linux Three-level scheme –Middle directories "collapsed" w/ macro tricks –No bits assigned for middle directory On context switch (address space change) –Save/load registers in TSS –Install new top-level directory (write cr3) –Flush TLB Lot's of macros for: –Allocating/deallocating, altering, querying… –Page directories, tables, entries –Examples: set_pte(), mk_pte(), pte_young(), new_page_tables()

Kernel Physical Layout Kernel is "pinned" (non-swappable) Usually about 1M; mapped in first 2M Some "holes" in low memory –Zero frame always invalid (avoids null dereference) –ISA "i/o aperture" (640K.. 1M) Kernel layout symbols (System.map) –_text (code start) (usually about 1M) –_etext (initialized data start) –_edata (uninitialized data start) –_end (end of kernel data) Remaining frames allocated as needed by kernel page allocator

Process Virtual Layout 4GB virtual space on a 32 bit system –Possible to install Linux on systems with more than 4GB of physical memory but you must use tricks to access "high" memory Kernel macro PAGE_OFFSET –Division between user and kernel regions –Typically 3GB user, 1 GB kernel (adjustable) We will look at details of user space later

Kernel Virtual Layout Kernel page tables initialized in two stages during startup –Phase one: only maps 4MB (provisional) –Phase two: maps all memory Provisional –Static PGD, just one PT –swapper_pg_dir –Low 4M of user & kernel map low 4M physical –Allows kernel to be addressed virtually or physically

Final Kernel Virtual Layout Kernel code –Operates using linear (virtual) addresses –Macros to go from physical to virtual and back _pa(virual), _va(physical) Kernel mapped using jumbo pages Kernel virtual address space –Low physical frames (containing kernel) mapped to virtual addresses starting at PAGE_OFFSET –Remaining frames allocated on demand by kernel and user processes and mapped to virtual addresses

Summary Intel hybrid segmented/paged architecture Segment support used minimally Linux has a conceptual 3 level page table Kernel is mapped into high memory of every process address space but is stored in low physical memory