CSE 5343/7343UNIX Case Study1 CSE 5343/7343 Fall 2006 Case Studies UNIX

CSE 5343/7343UNIX Case Study2 UNIX Case Study Outline History Design Philosophy Process Management –Processes –PCB/Process Table/U Area –Process States –Process Scheduling/Priorities –IPC Memory Management File Systems

CSE 5343/7343UNIX Case Study3 History ([2],[3]) 1965 – Bell and GE joined project MAC at MIT to develop Multics. Goal: multicomputing and data sharing for a large group of users. Ken Thompson (Bell) developed a game called “Space Travel” and found an unused PDP –Thompson and Ken Ritchie implemented an earlier designed file system on PDP-7. –This grew into UNIX. – File system design and idea of command interpreter (shell) as a user process came from Multics – Fork idea came from Berkeley’s GENIE OS.

CSE 5343/7343UNIX Case Study4 History (cont’d) 1971 – UNIX used in first real project: text processing for BELL labs patent department – UNIX required 16K bytes for system, 8K bytes for user programs, 512K bytes of disk, and limit of 64K disk bytes per file. Thompson set out to write a new Fortran compiler but developed a new programming language: B. B was based on BCPL (a tool for compiler writing and systems programming). B was interpretive. He later improved on B and called it C – UNIX rewrittten in C (unheard of at the time). AT&T offered UNIX free to universities – Ritchie and Thompson paper describing UNIX in CACM.

CSE 5343/7343UNIX Case Study5 History (cont’d) Bell combined several versions into UNIX System II and marketed. History (cont’d) 1978 – After distribution of Ver7 in 1978, the Unix Support Group (USG) at AT&T took over responsibilities from the research for distribution within AT&T – First external distribution from USG – System III. UC Berkeley developed their version of UNIX (Berkeley software Distributions). – BSD introduced vi in 2BSD, demand-pages virtual memory in 3BSD, TCP/IP networking protocol in 4.2BSD. – Less than 3% of BSD written in assembly – Finnish student Linus Torvalds wrote Linux for 80386, 32 bit processor

CSE 5343/7343UNIX Case Study6 Design Philosophy ([2],[3]) Smplicity C Time-sharing Simple user interface (modular and may be replaced) Device independence (treat files and devices in same manner) Aimed for programming environment Flexibility

CSE 5343/7343UNIX Case Study7 Design (cont’d) Programs such as shell and vi interact with kernel using well defined system call procedures. Cc built on top of c preprocessor, two-pass compiler.

CSE 5343/7343UNIX Case Study8 UNIX Process Management

CSE 5343/7343UNIX Case Study9 Processes Process is program in execution Consists of machine instructions (text), data, and stack regions. Separate stack for user and kernel mode. PID (Process ID) Processes are either user processes, daemon processes, or kernel processes. Daemons are not associated with user, but do system wide functions. Init may create daemons that exist throughout the life of the system or as needed.

CSE 5343/7343UNIX Case Study10 Solaris Threads ([3]) Kernel and user level threads Only kernel level threads are scheduled Implements Pthread API LWP between kernel and user level threads Each LWP associated with a kernel thread. User processes have at least one LWP. User threads may be bound or unbound to a LWP. May have kernel thread without LWP. Pool of LWPs for a process

CSE 5343/7343UNIX Case Study11 Solaris Threads (cont’d) User level thread –Thread ID –Registers –Stack pointer/stack –Priority Process –Process ID –Memory map –Open files –Priority –LWPs Kernel thread –Copy of kernel registers –Pointer to LWP –Priority –Scheduling information –Stack LWP –Registers –Memory –Accounting information

CSE 5343/7343UNIX Case Study12 PCB ([1],[2],[4]) Process Table: –State –UID for owner –Memory status (swapped or in memory) –Parent PID –Child PID –Event descriptor if suspended Entry in kernel process table that points to process region table. These in turn point to entries in the region table and to the regions for that process. This allows independent processes to share regions.

CSE 5343/7343UNIX Case Study13 U (User) Area ([2]) Information needed when process is executing. Process table slot Information about current system call File descriptors for open files Current directory Current root Login terminal Kernel has direct access to U area of currently executing process.

CSE 5343/7343UNIX Case Study14 Process States ([1],[2]) No context switch is needed to go from state 2 to state 1. States 3 and 7 are really the same. I/O request puts process to sleep. Zombie state – Child has finished execution, but parent wants to get information about it from the PCB.

CSE 5343/7343UNIX Case Study15 Process Hierarchy ([1]) Initial boot process (0) forks a child (process 1) and process 0 becomes the swapper. Process 1 is init process and is ancestor of all other processes. Parent/Child/Sibling relationship indicated by pointers in process table. Execve usually called to replace memory portion of new process. Exit – process termination Wait – Parent waits for child to exit. Reclaims process resources. Process group ID (GID) in process table Execute: ps -alx

CSE 5343/7343UNIX Case Study16 Process Scheduling([2],[3],[5]) Typical time slice values: –4.3BSD: 1/10 second –System V: times per second Round robin multilevel feedback queue At end of context switch, kernel executes algorithm to schedule a process. Highest priority process which is ready is scheduled. On tie, picks the one which has waited the longest.

CSE 5343/7343UNIX Case Study17 Process Scheduling (cont’d) Higher number – Lower Priority User and kernel priorities; user below kernel. User mode priority is function of recent CPU usage. Lower priority if recently used CPU. Priority assigned based on process group

CSE 5343/7343UNIX Case Study18 Process Scheduling (cont’d) Kernel to user mode: change to user mode and calculate based on kernel resources just used. Clock Handler adjusts all user mode process priorities at 1 second (System V) intervals and causes kernel to reschedule. Decay function adjusts recent CPU usage values at this time: decay(CPU) = CPU/2 Priority is recalculated at these 1 second intervals plus when a process is in the preempted but ready to run state: priority = CPU/2 + base level priority Base level priority is the threshold between user and kernel mode. Processes move up in queue but can not change to kernel mode.

CSE 5343/7343UNIX Case Study19 IPC ([2],[3]) Pipes – Only used from descendents of process that created the pipe. Named Pipes- Used by unrelated processes. Has a directory entry and is accessed as a file. Signals –Inform process of occurrence of asynchronous events. –Handling of signal by user process. Address of this routine is located in uarea next to signal number. Socket –Introduced by 4.2BSD –Transient object; Exists only as long as some process holds a descriptor referring to it. –Created by socket system call

CSE 5343/7343UNIX Case Study20 IPC (cont’d) Messages (System V) –On sending a message, a new entry is added to the associated linked list and the message is copied form the uarea. Message headers arranged in FIFO order. –Kernel awakens processes waiting for a message from this queue. –On receiving a message, process indicates what should be done if no message on the queue.

CSE 5343/7343UNIX Case Study21 IPC (cont’d) Shared Memory (System V) –System call creates a new region of shared memory or returns a current one. –Reading and writing is done as normal – no system calls. –Shared memory remains in tact even if no processes include it in their virtual address space.

CSE 5343/7343UNIX Case Study22 IPC (cont’d) Semaphores (System V) –System calls create and use semaphore. –Semaphore array where each entry is the count value. –If process is put to sleep, it sleeps at an interruptable priority and wakes up on receipt of a signal. –Handled similar to messages with id being the entry in the semaphore table.

CSE 5343/7343UNIX Case Study23 UNIX Memory Management

CSE 5343/7343UNIX Case Study24 Memory Management ([1],[2]) Depends on platform and version Early versions –Swapping –Transferred entire processes 3BSD – First demand-paged VM

CSE 5343/7343UNIX Case Study25 Swapping ([2]) Kernel swaps out a process if more memory space needed: –Fork system call must allocate space for child –Size of process increases –Kernel wants to free memory for processes previously swapped out Picking target process: –Zombie are not swapped (they take up no memory) –Sleeping swapped out before ready to run –Depends on Priority and time in memory –If no sleeping process then ready to run based on nice value and time in memory

CSE 5343/7343UNIX Case Study26 Swapping (cont’d) Nice Value used to change to scheduling priority Process 0 – Swapper –Highest scheduling priority Problem – thrashing:

CSE 5343/7343UNIX Case Study27 Demand Paging – System V [2] Approximate working set Additional Page Table entries Valid –Reference –Modify –Copy on Write – New copy must be created if updated –Age – Time in working set –Disk Block Descriptor Location on disk (VM) Demand fill – Immediately overwrite contents during exec Demand zero – Clear contents

CSE 5343/7343UNIX Case Study28 Address Translation (4.3BSD Vax)[1] Each region of address space mapped to a separate pag etable Separate User and system area page tables Two-level mapping, first level page tables reside in virtual memory Page is 512 bytes Virtual address –First two bits indicate regions 10 – system 01 – P0 - stack (user stack, kernel stack) 00 – P1 - Heap, data, text –User translation requires two levels of translation

CSE 5343/7343UNIX Case Study29 Page Replacements – Clock (4.3) [1]

CSE 5343/7343UNIX Case Study30 Memory Clock Algorithm Example Backing Store Process 1 Process 2 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid FrameFrame UpdateUpdate Disk Disk ValidValid Frame Table FreeFree PIDPID RefRef

CSE 5343/7343UNIX Case Study31 Memory Start State Backing Store Frame Table FreeFree PIDPID RefRef

CSE 5343/7343UNIX Case Study32 Memory Create Process 1 / Load into VM Backing Store Process 1 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid Frame Table FreeFree PIDPID RefRef ^^^^^^^^ ABCDABCD

CSE 5343/7343UNIX Case Study33 Memory Load 1 st Two Pages of P1 into Memory Backing Store Process 1 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid Frame Table FreeFree PIDPID RefRef ABCDABCD ^^^^^^^^ A B A B 1111

CSE 5343/7343UNIX Case Study34 Memory P1 Begins Executing using Pages 0&1 Backing Store Process 1 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid Frame Table FreeFree PIDPID RefRef ABCDABCD ^^^^^^^^ A B A B

CSE 5343/7343UNIX Case Study35 Memory Create Process 2 / Load into VM Backing Store Process 1 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid Frame Table FreeFree PIDPID RefRef ABCDABCD ^^^^^^^^ A B A B Process 2 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid DEFDEF ^^^^^^

CSE 5343/7343UNIX Case Study36 Memory Load 1 st Two Pages of P2 into Memory Backing Store Process 1 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid Frame Table FreeFree PIDPID RefRef ABCDABCD ^^^^^^^^ E AD B E AD B Process 2 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid DEFDEF ^^^^^^

CSE 5343/7343UNIX Case Study37 Memory P2 Begins Executing using Pages 0&1 – Update Page 1 Backing Store Process 1 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid Frame Table FreeFree PIDPID RefRef ABCDABCD ^^^^^^^^ E AD B E AD B Process 2 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid DEFDEF ^^^^^^

CSE 5343/7343UNIX Case Study38 Memory P2 Page Fault for Page 2 – P1 Executes During Paging I/O Backing Store Process 1 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid Frame Table FreeFree PIDPID RefRef ABCDABCD ^^^^^^^^ E AD B F E AD B F Process 2 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid DEFDEF ^^^^^^

CSE 5343/7343UNIX Case Study39 Memory I/O Completion Interrupt – P2 Begins Execution - Updates Page 2 Backing Store Process 1 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid Frame Table FreeFree PIDPID RefRef ABCDABCD ^^^^^^^^ E AD B F E AD B F Process 2 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid DEFDEF ^^^^^^

CSE 5343/7343UNIX Case Study40 Memory Timer Interrupt – P1 Begins Execution – Page Fault for Page 2 – No Free Frames – Initiate Clock Algorithm for Page Replacement Process 1 E AD B F E AD B F Frame Table FreeFree PIDPID RefRef Backing Store Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid ABCDABCD ^^^^^^^^ Process 2 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid DEFDEF ^^^^^^

CSE 5343/7343UNIX Case Study41 Memory Replace Frame 0 – Must swap out to disk as updated Backing Store Process 1 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid Frame Table FreeFree PIDPID RefRef ABCDABCD ^^^^^^^^ C AD B F C AD B F Process 2 Address Space PT FrameFrame UpdateUpdate Disk Disk ValidValid DEFDEF ^^^^^^

CSE 5343/7343UNIX Case Study42 References 1.Samuel J. Leffler, Marshall Kirk McKusick, Michael J. Karels, and John S. Quarterman, The Design and Implementation of the 4.3 BSD UNIX Operating System, Addison-Wesley, Maurice J. Bach, The Design of the UNIX Operating System, Prentice Hall, Abraham Silberschatz, Peter Baer Galvin, and Greg Gagne, Operating System Concepts Sixth Edition, John Wiley & Sons, Milan Milenkovic, Operating Systems Concepts and Design Second Edition, McGraw Hill, Andrew S. Tanenbaum, Modern Operating Systems, Prentice Hall, 1992.