Introduction Operating Systems (234123) – Spring 2013 Introduction Dan Tsafrir (11/3/2013) OS (234123) - spring intro1

Admin – staff Lecturer in charge – Prof. Dan Tsafrir Lecturer – Dr. Leonid Raskin TA in charge – Mr. Omri Azencot TAs – Ms. Anastasia Braginsky – Mr. Raja Giryes – Mr. Ilya Lesokhin – Mr. Alex Kreimer Homework checker – Mr. Hassan Abassie OS (234123) - spring intro2

Admin – course material & grading Course website – – Contains all course materials, e.g., office hours times & locations Lecture material – Updated on the fly, possibly significantly – Updates will be published as soon as possible Grades – Exam: 65% - 75% Must be >= 55 to pass – HW: 30% - 35% 4 wet, 4 dry, uneven weights, takef Not allowed to transfer grades from previous semester OS (234123) - spring intro3

Admin – books “Operating System Concepts” – By A. Silberschatz, P.B. Galvin, and G. Gagne – 9 th edition (>= 6 is fine) “Understanding the Linux Kernel” – By Daniel P. Bovet & Marco Cesati – 3 rd edition “Notes on Operating Systems” – By Dror G. Feitelson – Available for download via course website – This lecture: Chapters 1, Appendix A, and some of Chapter 2 A few others are listed in website, to those interested OS (234123) - spring intro4

LET US BEGIN Having gone over the admin stuff OS (234123) - spring intro5

What’s an operating system? A piece of software (SW) acting as an intermediary between user applications (apps) and the computer hardware (HW) First piece of SW to run on a computer when it’s booted Its job – Coordinate the execution of all SW, mainly user apps – Provide various common services needed by users and apps Common to draw it like so – But this is an oversimplification… OS (234123) - spring intro6 user application OS HW

What’s an OS? OS (234123) - spring intro7 signals It’s more complex…

physical HW What’s an OS? OS (234123) - spring intro8 It’s even more complex… Windows 8 “guest” virtual machine (VM) Windows XP “guest” VM Linux “guest” VM app “host” OS (called “hypervisor”) virtual HW The hypervisor (which is also an “OS”) can be: Linux / KVM, Vmware Workstation / Windows, Mac OS X / Parallels, … Topic is called “virtualization”

For simplicity Unless stated otherwise, we will focus on “classic” OS setup 1.Non-virtualized OS (also called “bare metal” OS) 2.System with one CPU (CPU = processor core) Both assumptions – Becoming less and less relevant in today’s world – But we’ll manage So, under the above assumptions… OS (234123) - spring intro9

A possible sequence of system actions 1.The OS executes; schedules some application app#1 (=makes it run) 2.App#1 runs = CPU executes its (non-privileged) ops; OS remains uninvolved 3.HW clock interrupt fires, invoking OS’s “interrupt handler” routine quantum 4.Interrupt handler updates OS’s notion of time + invokes OS scheduler if app#1 has been running too long (it is said that app#1 exhausted it “quantum”) context switch 5.Scheduler chooses app#2 to run in place of app#1 => “context switch” system cal 6.App#2 runs (non-privileged ops => without OS involvement); after a while, app#2 performs a “system call” to, e.g., read from a file 7.The system call causes a “trap” into the OS; the OS initiates the I/O op (using some privileged ops); it then puts app#2 to sleep, to await the I/O completion, and it chooses app#3 to run in its place 8.App#3 runs… Note: Either OS or app are running, not both OS regains control when (1) HW interrupt fires or (2) sys call is invoked OS (234123) - spring intro10

The OS is reactive “Ordinary” programs – Get some input, do some processing, produce output, and terminate – Have a main function, – Which is (more or less) the one and only entry point for the program In contrast, the OS – Waits (and reacts) to events – No main, but rather multiple points of entry, one per event type – Doesn’t terminate; when one event is finished, wait for the next – Goal: handle events as quickly as possible and allow apps to run – Q: why doesn’t the OS get stack? Events can be classified into two – HW device interrupts (keyboard, clock, hard drive, network card,...) – Systems calls (apps explicitly request service) OS (234123) - spring intro11

The OS performs “resource management” Example – Multiprogramming is one of the main features of OSes – Allows multiple apps to execute “at the same time” on a single CPU – To implement such concurrent execution, OS juggles the system’s resources between the competing apps – Trying to make it look as if each one has the computer for itself resource management – At the heart of multiprogramming lies “resource management”: Deciding which running app will get which resources Resources to manage – Obvious ones (taking the view of an app): CPU, memory, disk, NIC, keyboard, mouse, screen display – But also resources internal to the OS, e.g.: Disk space for “paging” (provide illusion of a huge memory) System tables (running apps, open files, network channels, etc.) OS (234123) - spring intro12

The OS provides “services” The OS provides services to running apps, e.g., – The ability to store data in files – The ability to communicate with remote machines – The ability to communicate with another app via shared memory – The ability to draw on screen – … Each service involves two aspects 1.Abstraction – provides convenience & portability by: (i) offering more meaningful, higher-level interfaces, and by (ii) hiding HW details, e.g., Apps use “files” instead of handing numerous raw block devices 2.Isolation – allow many apps to co-exist using the same HW devices without falling over each other’s feet, e.g., Several apps send/receive data over a network; OS keeps the data streams separated from each other OS (234123) - spring intro13

OS service: “abstract machines” Abstract machines simplify – The dynamics of multiprogramming are complex – Each app runs for a while, then preempted, then runs again, and so on – OS presents an abstraction that hides the complexity – Each app sees an abstract machine (seemingly) dedicated to itself – Apps are unaware of the OS activity Abstract machines isolate – Multiprogramming means there are many abstract machines, not one Each app gets its own abstract machine – Abstract machines are isolated from each other The abstraction hides all resources used to support other apps Thus, each apps sees the system as if it were dedicated to itself, and the OS juggles resources to support this illusion – Notably, the computation is still correct despite the multiprogramming OS (234123) - spring intro14

Abstraction allows for decoupling from physical restrictions – Abstract machine presented by OS is “better” than physical HW – By virtue of supporting more convenient abstractions Apps don’t access physical resources directly level of indirection – There’s a level of indirection – Otherwise how would we support, say, a 4GB of (virtual) memory with only, say, 1GB of (physical) memory? OS (234123) - spring intro15 Pros of abstraction – a “better” machine abstract

Pros of abstraction – portability Portability – The ability to correctly run the same exact program in different environments (in terms of source code, not binary) Portability across different HW – PowerPC (IBM), x86 (Intel, AMD), and SPARC (Oracle) chips support virtual memory completely differently – Namely, the aforementioned indirection is completely different – Linux runs on both type of chips, handling the HW diff – No need to rewrite user apps so as to cope with HW diff Portability across different OSes – There are lots of UNIX variants (Linux, HPUX, Solaris, *BSD, AIX,…) – All (more or less) implement the POSIX interface POSIX = “Portable Operating System Interface” – An app that obeys POSIX will run across all of these OSes unchanged OS (234123) - spring intro16

Three different ways to view an OS 1.By its programming interface – Its system calls 2.According to the services it provides – Time slicing (how multiprogramming is implemented as explained earlier), a file system, etc. 3.According to its internals, algorithms, and data structures An OS is defined by its interface – Different implementation of the same interface are equivalent as far as users and apps are concerned But the lectures are organized according to services – For each one we’ll detail the internal structures & algorithms used – Occasionally, we’ll provide examples of interfaces, mainly from Unix OS (234123) - spring intro17

Interaction between subsystems OS components can be studied in isolation; we’ll focus on – Process handling Processes are the agents of processing; the OS creates them, schedules them, and coordinates their interactions – Memory management Memory is allocated to processes as needed, but there might not be enough for all, so paging is used – File system Files are an abstraction providing named data repositories based on disks that store individual blocks; the OS does the bookkeeping OS (234123) - spring intro18

Interaction between subsystems But OS components typically interact, e.g., – OS improves utilization by overlapping computation of one process with the read ops of another – Likewise, if a process requires a memory page that currently resides on disk (due to paging), it suffers a page fault, which results in an I/O op; another process is run in the meanwhile. – Memory availability may determine if a new process is started or made to wait We’ll initially ignore such interactions to keep things simple OS (234123) - spring intro19

Scope – focusing on the kernel All the things we mentioned so far relate to the operating system “kernel” – Which will indeed be our focus But when one talks of an OS, one is typically referring to a “distribution”, which contains the following elements: – The Unix kernel itself; strictly speaking, this is “the OS” – The libc library; provides runtime environment for programs written in C (e.g., contains the implementation of printf, strcpy, etc.) – Various tools, such as gcc, the GNU C compiler – Many useful utilities/programs you may need, e.g., windowing system, desktop, and shell We’ll focus exclusively on the kernel — what it is supposed to do, and how it does it OS (234123) - spring intro20

HW SUPPORT FOR THE OS OS (234123) - spring intro21

Some HW mechanisms to support the OS HW clock Atomic synchronization operations Interrupts System calls Memory virtualization & protection I/O controllers – DMA (direct memory access) – Memory-mapped I/O – IOMMU (I/O memory management unit) Protected mode Privileged operations (instructions) OS (234123) - spring intro22

Privileged operations & privileged mode Some sensitive operations can only be invoked by the OS – All operations related to I/O – All operations related to memory indirection layers – Blocking/unblocking interrupts (why would we want to do that) – Memory usage bits – … HW supports at least 2 privilege modes – Kernel mode (ring 0 on x86) Kernel runs in this mode – User mode (ring 3 on x86) User apps run in this mode – Mode is saved in a “status bit” in a special register Which is accessible in kernel mode only – Privileged operations work only in kernel mode OS (234123) - spring intro23

Reading from disk on Linux/x86 User app invokes ‘read’ system call – read(int fileDescriptor, char *buffer, size_t bufferSize) libc does the following (in user mode) – Assigns the number ID associated with ‘read’ into register ‘eax’ – Prepares the other parameters (according to some convention) – Executes: int 0x80 (called “trap”) A synchronous interrupt (“int”) initiated by SW (0x80 in hexadecimal = 128 in decimal) Control transfers to kernel – HW changes mode from unprivileged (ring 3) to privileged (ring 0) – HW accesses IDT (interrupt descriptor table) array in entry 128 – IDT contains pointer-to-functions (addresses to kernel code) – Content of IDT is set by OS at boot time; user code can’t access it – Entry 128 of IDT points to the kernel’s “system call handler” routine – HW invokes this handler OS (234123) - spring intro24

Reading from disk on Linux/x86 Within the kernel – The system call identifier (‘read’) is used in a big switch statement to find and call the appropriate OS function that actually perform the desired service (‘sys_read’). – The latter function then starts by retrieving/validating its arguments from where they were stored by the wrapper libc function – It then generates an I/O read request that will be processed by the disk using DMA (no need to further involve the CPU) – OS then suspends the user app until the I/O read is completed – OS resumes another ready-to-run app I/O read is completed (a few milliseconds later) – An interrupt is generated, invoking the matching handler from IDT – OS figures out that the app’s I/O is completed, – OS changes app’s status from ‘waiting’ to ‘ready to run’ – When app is scheduled next, it will resume from after “int 0x80” OS (234123) - spring intro25

PROCESSES OS (234123) - spring intro26

מערכות הפעלה ( אביב 2009) חגית עטיה © 27 תהליכים  מהו תהליך ?  מבני הנתונים לניהול תהליכים.  החלפת הקשר.  ניהול תהליכים ע " י מערכת ההפעלה.

מערכות הפעלה ( אביב 2009) חגית עטיה ©28 תהליך  אבסטרקציה : יחידת הביצוע לארגון פעילות המחשב יחידת הזימון לביצוע במעבד ע " י מערכת ההפעלה תוכנית בביצוע ( סדרתי = פקודה - אחת - אחר - השנייה ) נקרא גם job, task, sequential process

מערכות הפעלה ( אביב 2009) חגית עטיה ©29 מה מאפיין תהליך ? תהליך לעומת תוכנית :  תוכנית היא חלק ממצב התהליך  אפשר לייצר כמה תהליכים מאותה תוכנית ( שירוצו במקביל )  מרחב כתובות  קוד התוכנית  נתונים  מחסנית זמן - ביצוע  program counter  רגיסטרים  מספר תהליך (process id)

מערכות הפעלה ( אביב 2009) חגית עטיה ©30 מרחב הכתובות של התהליך 0x xFFFFFFFF address space code (text segment) static data (data segment) heap (dynamic allocated mem) stack (dynamic allocated mem) Program Counter Stack Pointer

מערכות הפעלה ( אביב 2009) חגית עטיה ©31 מצבי התהליך  כל תהליך נמצא באחד המצבים הבאים : מוכן (ready) רץ (running) מחכה (waiting)  מתי מזמנים / מפנים תהליכים ? New Terminated Ready Running Waiting create kill/finish I/O, page fault, etc. I/O done resume preempt טווח קצר טווח בינוני / ארוך

מערכות הפעלה ( אביב 2009) חגית עטיה ©32 מבני הנתונים של תהליך ב Linux, מכיל O(100) שדות !  בכל נק ' זמן יש הרבה תהליכים פעילים במערכת  לכל תהליך מצב  Process control block (PCB) שומר את מצב התהליך כאשר אינו רץ. נקרא Process Descriptor ב -Linux  נשמר כאשר התהליך מפונה, נטען כאשר התהליך מתחיל לרוץ. Process id (PID) Execution state Program counter Stack pointer Registers Username Scheduling priority Memory limits

מערכות הפעלה ( אביב 2009) חגית עטיה ©33 מצב המעבד וה -PCB  כאשר תהליך רץ, המצב שלו נמצא במעבד : PC, SP, רגיסטרים רגיסטרים לקביעת גבולות זיכרון  כאשר המעבד מפסיק להריץ תהליך ( מעבירו למצב המתנה ), ערכי הרגיסטרים נשמרים ב -PCB.  כאשר המעבד מחזיר תהליך למצב ריצה, ערכי הרגיסטרים נטענים מה -PCB. Context Switch: העברת המעבד מתהליך אחד לשני.

מערכות הפעלה ( אביב 2009) חגית עטיה ©34 Context switch תהליך 1 תהליך 2 מערכת ההפעלה מתבצע מוכן ( אולי אחרי תק ' המתנה ) מוכן מתבצע save state in PCB 1 load state from PCB 2 save state in PCB 2 פסיקה או המתנה load state from PCB 1

מערכות הפעלה ( אביב 2009) חגית עטיה ©35 תורי מצבים  מערכת ההפעלה מחזיקה תורים של תהליכים תור ( או מבנה נתונים מתוחכם יותר ) ready תור waiting, למשל ל -device מסוים Wait queue header cat pcbfirefox pcb head pointer tail pointer Ready queue header head pointer tail pointer firefox pcbemacs pcbls pcb

מערכות הפעלה ( אביב 2009) חגית עטיה ©36 ה -PCB ותורי המצבים ה -PCB הוא מבנה נתונים בזיכרון מערכת ההפעלה.  כאשר התהליך נוצר, מוּקצה עבורו PCB ( עם ערכי התחלה ), ומשורשר לתור המתאים ( בדרך - כלל, ready ).  ה -PCB נמצא בתור המתאים למצבו של התהליך.  כאשר מצב התהליך משתנה, ה -PCB שלו מועבר מתור לתור.  כאשר התהליך מסתיים, ה -PCB שלו משוחרר.

מערכות הפעלה ( אביב 2009) חגית עטיה ©37 יצירת תהליך  תהליך אחד ( האב ) יכול ליצור תהליך אחר ( הבן ) שדה ppid בביצוע ps -al ב -Linux.  בדרך - כלל, האב מגדיר או מוריש משאבים ותכונות לבניו. ב -Linux, הבן יורש את שדה user, ועוד.  האב יכול להמתין לבנו, לסיים, או להמשיך לרוץ במקביל. רק תהליך אב יכול להמתין לבניו ב Windows יש קריאת CreateProcess( …,prog.exe, … ) מייצרת תהליך חדש ( ללא קשרי אבות / בנים ) אשר מתחיל לבצע את prog.

מערכות הפעלה ( אביב 2009) חגית עטיה ©38 יצירת תהליכים ב -UNIX:fork() int main(int argc, char **argv) { int child_pid = fork(); if (child_pid == 0) { printf( “ Son of %d is %d\n ”, getppid(),getpid()); return 0; } else { printf( “ Father of %d is %d\n ”, child_pid,getpid()); return 0; } }  יוצר ומאתחל PCB.  מיצר מרחב כתובות חדש, ומאתחל אותו עם העתק מלא של מרחב הכתובות של האב.  מאתחל משאבי גרעין לפי משאבי האב ( למשל, קבצים פתוחים ) באג נפוץ הוא שכיחת סגירת קבצים פתוחים של האב אצל הבן  שם את ה -PCB בתור המוכנים  עכשיו יש שני תהליכים, אשר נמצאים באותה נקודה בביצוע אותה תוכנית.  שני התהליכים חוזרים מה fork: הבן, עם ערך 0 האב, עם מספר התהליך (pid) של הבן נראה בהמשך מימוש יעיל יותר ל fork()

מערכות הפעלה ( אביב 2009) חגית עטיה ©39 איך מפעילים תוכנית חדשה ? int execv(char *prog, char **argv)  עוצר את ביצוע התוכנית הנוכחית.  טוען את prog לתוך מרחב הכתובות.  מאתחל את מצב המעבד, וארגומנטים עבור התוכנית החדשה.  מפנה את המעבד ( ה -PCB מועבר לתור המוכנים ). לא יוצר תהליך חדש !

מערכות הפעלה ( אביב 2009) חגית עטיה ©40 דוגמא : UNIX shell int main(int argc, char **argv) { while (1) { char *cmd = get_next_command(); int child_pid = fork(); if (child_pid == 0) { execv(cmd); fprintf(stderr, “ exec failed! ” ); } else { wait(child_pid); }