1. Sogang University Advanced Operating Systems (Process Management - Linux) Advanced Operating Systems (Process Management - Linux) Sang Gue Oh, Ph.D. Email : sgoh@macroimpact.com Email : sgoh@macroimpact.com

2. Sogang University Process Management - Linux Page 2 Process Descriptor in Linux Process Descriptor in Linux nEach process is represented by a task_struct data structure. nRather complex. It not only contains many fields, but pointers to other structures in order to manage processes.

3. Sogang University Process Management - Linux Page 3 Process State Process State n TASK_RUNNING : either executing on the CPU or waiting to be executed. n TASK_INTERRUPTIBLE : suspended (sleeping) until some condition becomes true. Can be interrupted by a signal or releasing a resource the process is waiting for. n TASK_UNINTERRUPTIBLE : similar to the previous state except that delivering a signal to the sleeping process leaves its state unchanged. n TASK_STOPPED : e.g., ctrl+z or debugging mode. TASK_ZOMBIE : a halted process but the parent process has not yet issued a wait() like system call to release the resource. This process still has a task_struct in the task vector.

4. Sogang University Process Management - Linux Page 4 Task Array (up to Linux 2.2.x) Task array u An array of pointers to every task_struct data structure. u Max. number of processes : the size of the task vector (default : 512 entries – defined with the variable NR_TASKS). u Linux 2.4 removes the task array to raise the hard-coded limit on the number of processes. Initializing task array #define NR_TASKS 512 #define init_task(init_task_union.task) struct task_struct *task[NR_TASKS] = { &init_task,} ; --- init_task is a process descriptor pointer to the process 0 or swapper process.

5. Sogang University Process Management - Linux Page 5 Storing a Process Descriptor The task array only contains pointers to process descriptors. n Process descriptors are stored in dynamic memory (alloc_task_struct()). n Linux stores two different data structure in a single 8 KB memory. n They are process descriptor and kernel mode stack for each processor. union task_union { struct task_struct task; unsigned long stack[2048]; } --- include/asm/processr.h

6. Sogang University Process Management - Linux Page 6 Accessing a Processor Descriptor Accessing a Processor Descriptor n The esp register is the CPU stack pointer. n Right after switching from user mode to kernel mode, the kernel stack of a process is always empty. n The process descriptor pointer of a process can be easily obtained via esp register. t Mask out the 13 least significant bits of esp. t This is done by the current macro.  The current macro contains the process descriptor pointer of currently running process (e.g., current->pid : pid of the process currently running). movl $0xffffe000, %ecx andl %esp, %ecx movl %ecx, p  current

7. Sogang University Process Management - Linux Page 7 Process List Process List n Process list for all existing processes n Process list of TASK_RUNNING state n Process list of task free entries

8. Sogang University Process Management - Linux Page 8 PID Hash Table PID Hash Table n Used to derive the process descriptor pointer via pid. n In order to speed up the search, a hash table (pidhash variable) consisting of PIDHASH_SZ elements (default is 128 in Linux 2.2.x, 1024 in Linux 2.4.x) has been introduced. n Hash function: #define pid_hashfn(x) ((((x) >> 8) ^ (x) & (PIDHASH_SZ – 1)) n Uses chaining to handle collision. pidhash 0 100 123 127 Pid 228Pid 27535 Pid 27536 pidhash_next pidhash_pprev

9. Sogang University Process Management - Linux Page 9 Parenthood Relationship t p_opptr : original parent t p_pptr : parent t p_cptr : youngest child t p_ysptr : younger sibling t p_osptr : older sibling t Every task_struct keeps pointers to its parent process and to its siblings as well as to its own child processes. pstree command /* * pointers to (original) parent process, youngest child, younger sibling, * older sibling, respectively. (p->father can be replaced with * p->p_pptr->pid) */ struct task_struct *p_opptr, *p_pptr, *p_cptr, *p_ysptr, *p_osptr;

10. Sogang University Process Management - Linux Page 10 Example of Parenthood Relationship

11. Sogang University Process Management - Linux Page 11 Wait Queues (include/linux/wait.h and sched.h) Wait Queues (include/linux/wait.h and sched.h) n Process list of processes with TASK_INTERRUPTIBLE or TASK UNINTERRUPTIBLE states. Wait queue pointerReceives the address q of a wait queue and set that pointer to q – 4. q struct wait_queue { struct task_struct *task; struct wait_queue *next; } add_wait_queue(struct wait_queue **q, struct wait_queue *entry); remove_wait_queue(struct wait_queue **q, struct wait_queue *entry);

12. Sogang University Process Management - Linux Page 12 Using Wait Queues Using Wait Queues sleep_on(struct wait_queue **p) : set process state to TASK_UNINTERRUPTIBLE and inserts the process into the wait queue. Can be waken up via wake_up macro. interruptible_sleep_on(struct wait_queue **p) : set process state to TASK_INTERRUPTIBLE and inserts the process into the wait queue. Can be waken up via wake_up_interruptible macro or receiving a signal. sleep_on_timeout(struct wait_queue **p, long time) : similar to sleep_on except that the process can be awakened via timeout. interruptible_sleep_on_timeout(struct wait_queue **p, long time) : similar to interruptible_sleep_on except that the process can be awakened via timeout. wake_up(struct wait_queue **p) : macro to wake up all the sleeping processes with TASK_UNINTERRUPTIBLE state in the wait queue. Invoke __wake_up(queue, mode) where mode is the process states for waking up. wake_up_interruptible(struct wait_queue **p) : macro to wake up all the sleeping processes with TASK_INTERRUPTIBLE state in the wait queue. Invoke __wake_up(queue, mode) function.

13. Sogang University Process Management - Linux Page 13 Process Usage Limits Process Usage Limits n Processes are associated with sets of usage limits, which specify the amount of system resources they can use. n They are stored in the rlim field of the process descriptor. n The field is an array of elements of type struct rlimit (e.g., struct rlimit rlim[]), t struct rlimit { /* define in include/linux/resource.h */ long rlim_cur; long rlim_max; } n Examples of elements are RLIMIT_CPU (maximum CPU time), RLIMIT_FSIZE (maximum file size), RLIMIT_NPROC (maximum # of processes that the user can own), etc. (defined in include/asm/resource.h ) n current->rlim[RLIMIT_CPU] : CPU time limit of current process.

14. Sogang University Process Management - Linux Page 14 Creating Processes Creating Processes n Traditional UNIX systems treat all processes in the same way: resources owned by the parent process are duplicated and a copy is granted to the child process. n This approach makes process creation very slow and inefficient. n Three approaches : t Copy on Write (COW) : allows both the parent and the child to read the same physical pages. Whenever either one tries to write on a physical page, the kernel copies its contents into a new physical page that is assigned to the writing process. fork() is implemented with COW. t Lightweight Processes : allows both the parent and the child to share many per-kernel data structure. t vfork() system call : creates a process that shares the memory address space of its parent. Parent’s execution is blocked until the child exits or executes a new program. In Linux, vfork() is identical to fork().

15. Sogang University Process Management - Linux Page 15 Copy on Write Copy on Write Physical Memory Parent PTChild PT Parent PT Physical Memory Shared Copy on Write copy Parent copy Child copy write

16. Sogang University Process Management - Linux Page 16 Creating Processes in Linux Creating a process ( kernel/fork.c - do_fork( )) t New processes are created by cloning old processes. l A new task_struct is allocated (alloc_task_struct() to get 8 KB). l Copies the contents of the parent’s task descriptor into the new descriptor. l Check the usage limit. l A new process identifier is created. l The new task_struct is entered into the task vector. l Updates all the process descriptor fields that cannot be inherited from the parent process, such as the fields that specify the process parenthood relationships. Create new data structures and copy into them the values of the corresponding parent data structures. Update the various process list and hash table.

17. Sogang University Process Management - Linux Page 17 Example (Process Creation – User Program) Example (Process Creation – User Program) #include main() {  int childpid;  if ( ( childpid = fork() ) < 0 ) {  printf(“can’t fork\n”);  exit(-1);  }  else if (childpid > 0) { /* parent process */  …..  Parent_Service_routine(); OR  execvp(“prog_name1”, argv); OR  while ( wait((int *) 0) != childpid) ;  …..  exit(0);  }  else { /* child process */  …..  Child_Service_Routine(); OR  execvp(“prog_name2”, argv);  …..  exit(0);  } }

18. Sogang University Process Management - Linux Page 18 Kernel Threads Kernel Threads n Since some of the system processes run only in Kernel Mode, modern operating systems delegate their functions to kernel threads (kernel-level lightweight process). n What is different from regular process ? t Each kernel thread executes a single specific kernel function, while regular processes execute kernel functions only through system calls. t Kernel threads run only in Kernel Mode, while regular processes run alternatively in Kernel Mode and in User Mode. t Since kernel threads run only in Kernel Mode, they use only linear addresses greater than PAGE_OFFSET. Regular processes, on the other hand, use all 4 GB of linear addresses, either in User Mode or in Kernel Mode. t Kernel threads share kernel data structures. t Both kernel threads and regular process occupy pid and the corresponding process descriptor. Created via kernel_thread(int (*fn) (void *), void *arg, unsigned long flags).

19. Sogang University Process Management - Linux Page 19 Process Address Space (Virtual Memory) Process Address Space (Virtual Memory) n The address space of a process consists of all linear addresses that the process is allowed to use. n Each process sees a different set of linear addresses. n The kernel dynamically modify a process address space by adding or removing intervals of linear addresses (memory region). n For reasons of efficiency, both the linear address and the length of a memory region must be multiples of 4K. n All information related to the process address space (memory descriptor) is included in a table referenced by the mm field of the process descriptor. n Each entry of the table contains the information of all the memory region within the process.

20. Sogang University Process Management - Linux Page 20 Memory Descriptor (mm_struct) Memory Descriptor (mm_struct) Pointers to the memory regions.

21. Sogang University Process Management - Linux Page 21 Memory Region (vm_area_struct) Memory Region (vm_area_struct) Start address of a memory region End address of a memory region Next memory region

22. Sogang University Process Management - Linux Page 22 Adding or Removing Memory Regions Adding or Removing Memory Regions

23. Sogang University Process Management - Linux Page 23 Memory Mapping (1) n The executable binary file of a process needs to be mapped into the virtual address space (4 GB) before executing. (Use do_mmap() function) n Procedure  Generate a set of vm_area_struct ( include/linux/mm.h). t Each vm_area_struct represents a part of the executable image. l The executable code, initialized data, uninitialized data (BSS), etc. l Associate the correct set of virtual memory operations. t All mapped areas (vm_area_struct) belonging to the same process are connected using a tree structure. l AVL tree structure - for efficient search (O(n) -> O(logn)). l n is typically around 6 but may reach 3000 in some cases.

24. Sogang University Process Management - Linux Page 24 Memory Mapping (2) mm count pgd mmap_avl mmap vm_end vm_start vm_ops vm_inode vm_flags vm_next vm_end vm_start vm_ops vm_inode vm_flags vm_next Data Code vm_area_structmm_struct Task_struct open() close() …. nopage() swapin() swapout() Virtual Memory Operations

25. Sogang University Process Management - Linux Page 25 Memory Mapping - Example n The vm_area_list of a process can be seen at /proc/pid/maps. n An example for the init process. $cat /proc/1/maps 08048000-0804e000r-xp00000000 08:03 52838 # /sbin/init - code 0804e000-0804f000rw-p00005000 08:03 52838 # /sbin/init - data 0804f000-08054000rwxp00000000 00:00 0 # bss 40000000-40012000r-xp00000000 08:03 36578 # /lib/ld-2.1.2.so - code 40012000-40013000rw-p00012000 08:03 36578 # /lib/ld-2.1.2.so - data 40018000-40103000r-xp00000000 08:03 36585 # /lib/libc-2.1.2.so - code 40103000-40107000rw-p000ea000 08:03 36585 # /lib/libc-2.1.2.so - data 40107000-4010b000rw-p00000000 00:00 0 # bss bfffe000-c0000000rwxpfffff000 00:00 0 # stack

26. Sogang University Process Management - Linux Page 26 Getting a New Memory Region Getting a New Memory Region n When the user types a command at the console -> new process. n A running process may decide to load an entirely different program. n A running process may perform a “memory mapping” on a file. n A process may keep adding data on its user mode stack. n A process may expand its dynamic area (heap) through a function call such as malloc(). n A process may create an IPC shared memory region.

27. Sogang University Process Management - Linux Page 27 Memory Region Access Rights (vm_flags) Memory Region Access Rights (vm_flags)

28. Sogang University Process Management - Linux Page 28 Memory Region Handling Memory Region Handling n Finding the closest region to a given address t find_vma() t find_vma_prev() n Finding a region that overlaps a given address interval t find_vma_intersection() n Finding a free address interval t get_unmapped_area() n Inserting a region in the memory descriptor list t insert_vm_struct() n Merging contiguous regions t merge_segments()

29. Sogang University Process Management - Linux Page 29 Overall Scheme for Page Fault Handler Overall Scheme for Page Fault Handler

30. Sogang University Process Management - Linux Page 30 Flow Diagram of the Page Fault Handler Flow Diagram of the Page Fault Handler

31. Sogang University Process Management - Linux Page 31 Demand Paging - Page Fault Handling (1) n do_page_fault() ( arch/i386/mm/fault.c ) - page fault handler t Find the vm_area_struct that the page fault occurred in (find_vma()). t If not found then illegal access, send SIGSEGV signal. else call handle_mm_fault(). handle_mm_fault() (mm/memory.c) t Search of the page table entry exists. t If no, allocate a new page table entry. t Call handle_pte_fault(). n handle_pte_fault() (mm/memory.c) switch (cause) { case “memory not present” : call do_no_page(); break; case “protection violation” : call do_wp_page(); break; case “swap out” : call do_swap_page(); break; }

32. Sogang University Process Management - Linux Page 32 Demand Paging - Page Fault Handling (2) n do_no_page() (mm/memory.c) t Allocate a new page, and update page table entry. n do_wp_page() (mm/memory.c) t Fault by copy-on-write. t Allocate a new page and copy the old page into a new page. t Decrease the map count of old page by 1. n do_swap_page() (mm/memory.c)  Load appropriate pages into memory from the swap area.

33. Sogang University Process Management - Linux Page 33 Page Cache Management (1) n Purpose t To speed up access to files on disk. t Store pages from memory mapped files. n Page Hash Table t A vector of pointers to mem_map_t. t Indexed by VFS inode and the file offset. t If page found, return the mem_map_t. else, allocate a physical page and read from the file. n Single Page Read Ahead t Accessing the pages in the file serially.

34. Sogang University Process Management - Linux Page 34 Page Cache Management (2) inode offset next_hash prev_hash page_hash_table mem_map_t inode offset next_hash prev_hash 12 0x2000 12 0x8000 struct page *page_hash_table[HASH_SIZE] ˀ when a read request for a page occurs - hash table is checked for the existence of that page - if (exist) no need to read from the file system

35. Sogang University Process Management - Linux Page 35 Swapping Out and Discarding Pages (1) Kernel Swap Daemon (kswapd - mm/vmscan.c ) t Kernel thread. t Ensure enough free pages in the system. t Started by the kernel init process at startup. t Kernel swap timer (periodically awaken - basically once a second). n Variables t free_pages_high t free_pages_low t nr_async_pages l number of pages waiting to be written to the swap file.

36. Sogang University Process Management - Linux Page 36 Swapping Out and Discarding Pages (2) n Swap Daemon Operation t If free pages > free_pages_high do nothing. else, try three ways to free physical pages. l Reducing the size of the buffer and page caches. l Swapping out System V shared memory pages. l Swapping out and discarding pages. t If free pages < free_pages_low, l Try to free 6 pages and sleep for half its usual time. t If free_pages_low < free pages < free_pages_high, l Try to free 3 pages.

37. Sogang University Process Management - Linux Page 37 Reducing the Size of the Caches Why shrink caches ? (mm/filemap.c) t Page cache and buffer cache entries are good candidates. t Relatively easy since we don’t need to write to physical devices. t All processes need to suffer equally. n Method t Examine a block of pages in the mem_map page vector in cyclical manner (clock algorithm). t If cached in either the page cache or the buffer cache remove from cache. t If all the buffers in a page are freed, then the page is also freed. (In case when the page itself is cached in the buffer cache)