Linux Memory Issues An introduction to some low-level and some high-level memory management concepts

Some Architecture History 8080 (late-1970s) 16-bit address (64-KB) 8086 (early-1980s) 20-bit address (1-MB) (mid-’80s) 24-bit address (16-MB) (late-’80s) 32-bit address (4-GB) (late-’90s) 36-bit address (64-GB) Core2 (mid-2000s) 40-bit address (1-TB)

‘Backward Compatibility’ Many buyers resist ‘early obsolescence’ New processors need to run old programs Early design-decisions leave their legacy 8086 could run recompiled 8080 programs 80x86 can still run most 8086 applications Win95/98 could run most MS-DOS apps But a few areas of incompatibility existed

Linux must accommodate legacy Legacy elements: hardware and firmware CPU: reset-address and interrupt vectors ROM-BIOS: data area and boot location Display Controllers: VRAM & video BIOS Support chipsets: 15MB ‘Memory Window’ DMA: 24-bit memory-address bus SMP: combined Local and I/O APICs

Other CPU Architectures Besides IA-32, Linux runs on other CPUs (e.g., PowerPC, MC68000, IBM360, Sparc) So must accommodate their differences –Memory-Mapped I/O –Wider address-buses –Non-Uniform Memory Access (NUMA)

Nodes, Zones, and Pages Nodes: to accommodate NUMA systems However 80x86 doesn’t support NUMA So on 80x86 Linux uses just one ‘node’ Zones: to accommodate distinct regions Three ‘zones’ on 80x86: –ZONE_DMA (memory below 16-MB) –ZONE_NORMAL (from 16-MB to 896-MB) –ZONE_HIGHMEM (memory above 896-MB)

Zones divided into Pages 80x86 supports 4-KB page-frames Linux uses an array of ‘page descriptors’ Array of page descriptors: ‘mem_map[]’ physical memory is ‘mapped’ by CPU

How 80x86 Addresses RAM Two-stages:‘segmentation’ plus ‘paging’: –First: logical address  linear address –Then: linear address  physical address CPU employs special caches: –Segment-registers contain ‘hidden’ fields –Paging uses ‘Translation Lookaside Buffer’

Logical to Linear selector GDTR segment-registeroperand-offset virtual address-space memory segment global descriptor table base-address and segment-limit descriptor

Segment Descriptor Format Limit[ ] Base[ ] Base[ ]Base[ ] Limit [ ] 310

Linear to Physical physical address-space offsettable-index linear address CR3 dir-index page frame page directory page table

CR3 and CR4 Register CR3 holds the physical address of the current task’s page-directory table Register CR4 was added in the so software would have the option of “turning on” certain advanced new CPU features, yet older software still could execute (by just leaving the new features “turned off”)

Example: Page-Size Extensions can map either 4KB or 4MB pages With 4MB pages: middle table is omitted Entire 4GB address-space is subdivided into MB-pages Demo-module: ‘cr3.c’ creates a pseudo-file showing the values in CR3 and in CR4

Linear to Physical physical address-space offset linear address CR3 dir-index page frame page directory 4-MB page-frames

PageTable Entry Format Frame Address 0 31 Frame attributes 1112 Some Frame Attributes: P : (present=1, not-present=0) R/W : (writable=1, readonly=0) U/S : (user=1, supervisor=0) D : (dirty=1, clean=0) A : (accessed=1, not-accessed=0) S : (size 4MB = 1, size 4KB = 0)

Visualizing Memory Our ‘pgdir.c’ module creates a pseudo-file that lets users see a visual depiction of the CPU’s current ‘map’ of ‘virtual memory’ Virtual address-space (4-GB) –subdivided into 4MB pages (1024 pels) –Text characters: 16 rows by 64 columns

Virtual Memory Visualization Shows which addresses are ‘mapped’ Display granularity is 4MB Data is gotten from task’s page-directory Page-Directory location is in register CR3 Legend: ‘-’ = frame not mapped ‘3’ = r/w by supervisor ‘7’ = r/w by user

Assigning memory to tasks Each Linux process has a ‘process descriptor’ with a pointer inside it named ‘mm’: struct task_struct { pid_tpid; char comm[16]; struct mm_struct*mm; /* plus many additional fields */ };

struct mm_struct It describes the task’s ‘memory map’ Where’s the code, the data, the stack? Where are the ‘shared libraries’? What are attributes of each memory area? How many distinct memory areas? Where is the task’s ‘page directory’?

Demo: ‘mm.c’ It creates a pseudo-file: ‘/proc/mm’ Allows users to see values stored in some of the ‘mm_struct’ object’s important fields

Virtual Memory Areas Inside ‘mm_struct’ is a pointer to a list Name of this pointer is ‘mmap’ struct mm_struct { struct vm_area_struct*mmap; /* plus many other fields */};

Linked List of VMAs Each ‘vm_area_struct’ points to another struct vm_area_struct { unsigned longvm_start; unsigned longvm_end; unsigned longvm_flags; struct vm_area_struct*vm_next; /* plus some other fields */};

Structure relationships VMA mm_struct task_struct *mm *mmap Linked list of ‘vm_area_struct’ structures The ‘process descriptor’ for a task Task’s mm structure

Demo ‘vma.c’ module It creates a pseudo-file: /proc/vma Lets user see the list of VMAs for a task Also shows the ‘pgd’ field in ‘mm_struct’ EXERCISE Compare our demo to ‘/proc/self/maps’

In-class exercise #2 Try running our ‘domalloc.cpp’ demo It lets you see how a call to the ‘malloc()’ function would affect an application list of ‘vm_area_struct’ objects NOTE: You have to install our ‘vma.ko’ kernel-object before running ‘domalloc’