1. CIS 191: Linux and Unix Class 6 March 4, 2015 Processes, Scheduling, and Devices

2. Outline Processes Concurrent Operation and Scheduling Unix Process Management Devices

3. What is a Program? We spend a lot of our time writing programs, but what are they, really?

4. What is a Program? We spend a lot of our time writing programs, but what are they, really? – Bits of code that run when we tell them to? – A collection of instructions to the machine? – Compiled machine code?

5. What is a Program? We spend a lot of our time writing programs, but what are they, really? – Bits of code that run when we tell them to? – A collection of instructions to the machine? – Compiled machine code? How do we run programs…?

6. Processes Processes are “programs in execution” – They have associated context (data and execution) – The execution must progress sequentially

7. Processes Processes are “programs in execution” – They have associated context (data and execution) – The execution must progress sequentially Processes are not programs or applications – We could run a program any number of times Each instance of a program is run as a process! Each process has its own address space and contexts

8. Why Processes? Operating systems (including Linux versions) have to execute a lot of different programs – And these programs have to run concurrently… Concurrent Processes

9. Why Processes? Operating systems (including Linux versions) have to execute a lot of different programs – And these programs have to run concurrently… Concurrent Processes – Word processor, browser, email all running at once – Internally, memory management processes – Device serving processes…

10. Processes are Separated Each process has its own context in which it executes In fact, each process believes it has access to the entire machine! – This isn’t true, of course – that would lead to disaster – We aren’t going to go into the details of how this is implemented in this class, but you’ll get a full dose of it in both CIS 380 (OS) and CIS 371 (computer organization) Yes, it’s sufficiently complicated to be fully covered in two classes

11. Process Contexts PC – Program Counter – address of next instruction Stack – temporary data – function parameters, return addresses, locals Heap – dynamically allocated memory – objects, strings Static Data – globals Code – text that’s executed

12. Process Creation Parent processes create children processes – This forms a tree of processes A Process is identified and managed through its process identifier (pid) – View processes on your machine with the ps command ps -a will allows you to see all processes (including the os’s)

13. Unix Process Hierarchy A Unix process is spawned by a parent process A Unix process can spawn any amounts of children The “root” process is called init, and it is created when the operating system is booted – So every process can be traced back to init Each process has a unique process ID number

14. Unix Shell and Processes When you type a command, the shell clones itself The child process then calls exec, which causes it to stop executing the shell and start executing your command The parent shell waits for the child to terminate

15. Recall the unix shell and subshells Call a command in parenthesis ( ) to execute the command in a subshell Call a command in curly braces { } to group them but not spawn a subshell Call a command ending with the & symbol to execute the command in a subshell in the background – Commands in the background can’t write to stdout until they terminate (finish)

16. Process Substitution Cool facility to get output from one process into another process Works by creating a special file descriptor which is passed to the command you are substituting to, which can then be read by a receiving command <(commands) subs the results out of the process – $ diff <(ls.) <(ls..) diff the contents of current dir with parent dir >(commands) subs a file into a process as input – $ ls > >(cat) substitute results of list into another process running cat

17. Keeping track of Processes The kernel’s job A data structure, usually called the process table, keeps track of each active process Entry in this table are called process control blocks (PCBs) So a process’ context is described by its PCB! – The kernel can stop and restart processes – In other words, the state of execution of the process is stored

18. Outline Processes Concurrent Operation and Scheduling Unix Process Management Devices

19. Process states Running – The process is currently executing. Only one process can be executing at a time on a single core. Blocked – The process is waiting – Perhaps for I/O? Ready – The process would like to run

20. How to schedule? We need to schedule processes so that the system runs “well” But what does “well” mean?

21. How to schedule? We need to schedule processes so that the system runs “well” But what does “well” mean? It doesn’t necessarily mean what’s best for any particular application – If you’re an application writer, you should consider the operating system to be a malicious adversary… – It makes it easier to write robust programs that wayf

22. We need to balance between… Efficiency – Spend as much time in user processes as we can Fairness – Avoid starving processes for compute time Priority Handling – Need to allow more important processes better service Real-time constraints – Need to have a guaranteed level of service Hardware constraints – How much time do we waste switching processes?

23. Take One : FCFS First Come First Serve – When a job arrives, place it on the end of the service queue – The dispatcher selects the first job and runs it to completion of a CPU burst It’s a nice, simple algorithm But it’s inappropriate for interactive systems… – Why?

24. Take two : Shortest Job First Ready queue is thought of as a priority queue, with the shortest job sorted to be first Then run like in FCFS This would have a provably optimal waiting time But starvation is possible And you can’t actually implement it on real computers – Would have to approximate based on CPU bursts

25. Take three : Round Robin First come first serve, but preemptive – So a process can be interrupted against its will We have a circular ready queue, similar to in FCFS Arriving jobs are placed at the end The dispatcher selects the first job in the queue and runs it to the completion of a CPU burst, or until the time quantum expires If the quantum expires, then the job is placed at the end

26. Take three : Round Robin This is a simple, low overhead approach that works for interactive systems But this relies on the time quantum value – If it’s too large (longer than a CPU burst), we approach FCFS – If it’s too small, we spend too much time in context switching – A typical value is around 20-40 ms A good rule of thumb is to choose a quantum so that the large majority of jobs finish their CPU burst in one quantum Round robin makes the assumption that all processes are equally important

27. Task priority But priority can be very important… – We want user-facing processes to be addressed first – Keyboard input, text processors How can we select/compute the priorities?

28. Multi-Level Feedback Queues Use the round robin technique, but have some levels of queues The time quantum gets bigger as you go “down” the queues; priority decreases as you go down Processes start in the highest level queue If a process is not complete at the end of a time quantum, it is put at the bottom of the next queue down A process in a given queue is not scheduled until all the higher queues are empty

29. What happens here? Processes that do a lot of I/O congregate in the higher level queues – Then don’t have to spend much time doing computation – Most of their time is spend waiting for devices Processes that do a lot of processing sink into lower queues, where they’ll have more time to execute (when they are scheduled) This will result in a more responsive system overall – Why?

30. Completely Fair Scheduler In practice, today most Linux distributions make use of the “Completely Fair Scheduler”, or CFS.“Completely Fair Scheduler”, or CFS This makes use of a red-black tree to sort processes which would like to execute based on the amount of time they spent in execution so far The process which has spent the least time in execution is what’s selected next For more information, check out the wikipedia page! (Linked above)

31. Outline Processes Concurrent Operation and Scheduling Unix Process Management Devices

32. Scheduling in Unix Based on the idea of multi-level feedback queues Priorities from -64 to 63 (lower = higher priority) Negative numbers are reserved for processes waiting to service devices Time quantum is.1 seconds – Empirically found to be the longest quantum which doesn’t reduce perceived responsiveness for editors, etc – But the longer quantum means less context switch Adjust priorities to reflect resource requirement (waiting for event) and resource consumption (CPU time)

33. Unix CPU Scheduler There are two values (stored in PCB/process table entry) – p_cpu: estimate of recent CPU usage – p_nice: a user/OS settable weighting factor (-20 -> 20) default is 0, negative is higher priority, positive is lower priority Each process’ priority is calculated periodically – priority = base + p_cpu + p_nice p_cpu is incremented each time the system clock ticks and the process is found to be executing

34. Manually managing processes You can view which processes are running by using the ps command – This can be piped to grep to find a particular process You can then stop a running process by sending signals with the kill command (it does more than kill…) – Really it sends a signal, based on what you pass it – kill -9 Sends the kill signal! This can’t be handled or interrupted. – kill -2 Sends SIGINT (same as ctrl-C) – And other things! Check out the man page for more details.

35. Viewing live process update feed You can run the top command to see a running update of the processes running on your machine Like running ps over and over again… You can also install htop, which is a tricked out version of top, through the apt-get.

36. Outline Processes Concurrent Operation and Scheduling Unix Process Management Devices

37. Remember /dev? /dev contains system device files Linux systems accomplish tons of magic by treating devices as special files – And by pretending that certain non-device objects are files… – Linux employs devices to provide lightweight “system services” The contents of /dev have odd permissions if you check with ls -l

38. Device Files are “Pseudofiles” When you read from or write to a device “file”, the operating system silently intercepts your request and feeds it to a driver The driver knows how to interact with the device Drivers are typically written in C – Can anyone tell me why that might be?

39. Device File Permissions If you took the liberty of running ls –l on the files, you might see something like this The “b” means that this is a “block” device You could also see a “c”, which would mean it is a “character” device The size field has been replaced by a csv field where – The first value represents the major device number – The second value represents the minor device number

40. Types of Devices Character Devices – Denoted by “c” character at start of permissions – Provide direct, unbuffered access to the hardware – Examples: Serial ports, keyboards, sound cards, modems Block Devices – Denoted by “b” character at start of permissions – Provide buffered access to the hardware – Always allowed to read/write any sized block – Buffered => We don’t know when the data will be written Pseudo Devices – Fake devices that are useful

41. Some Pseudo Devices /dev/null – Accepts all data written to it and does nothing – http://devnull-as-a-service.com/ http://devnull-as-a-service.com/ /dev/full – Always full; returns a continuous stream of NULL bytes when read and returns “disk full” error when written to /dev/zero – Produces endless string of zero bits when read /dev/random and /dev/urandom – Contains a stream of random bytes

42. Hard Disk Partitions Each computer may have several hard drives attached, and each hard drive can be divided into several partitions – Windows assigns each partition its own drive letter (like C:) – Linux allows you to specify where the data on a given partition should appear in the filesystem Every hard drive is assigned a name in /dev – Like /dev/sda for the first drive, or /dev/sdb for the second – Naming starts at sd followed by a letter The n th partition of the drive sdb is sdb(n) – So sdb3 is the third partition on the second hard disk

43. Mounting and Unmounting To use a partition, you can use the mount command – The usage is mount device location – For example, mount /dev/sda2 /media/windows The mounted filesystems and devices are tracked in /etc/mtab. You’ll probably need to be root to access it. umount unmounts a directory – Note the absence of the ‘n’ in umount

44. fstab Non-changing filesystem information is written in /etc/fstab – According to the man page, “it is the duty of the system administrator to properly create and maintain this file” – At boot, fstab tells the system which filesystems should be loaded – Afterwards, fstab is used (by mount/umount) to describe how to mount and unmount filesystems fstab entries contain the filesystem location, the mount point, the file system type, options, and information about core dumping and checking the filesystem