1. Processes and Threads Case studies: Windows and Linux Lecture 6 ~ Fall, 2007 ~

2. No. 2 Fall, 2007TUCN. Operating Systems. Lecture 6 Contents Windows 2000 process management Linux process management

3. No. 3 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes Fundamental concepts Jobs Each job has one or more processes Processes Each process has one or more threads Threads Kernel threads - each thread has one or more fibers Fiber User space threads

4. No. 4 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes Jobs A collection of one or more processes managed as a single unit Each job object has quotas and resource limits associated with it maximum number of processes CPU time available per process and per job maximum memory usage per process and per job security restrictions imposed on processes Win32 API CreateJobObject AssignProcessToJobObject SetInformationJobObject QueryInformationJobObject

5. No. 5 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes Processes Containers for resources A 4GB address space the bottom 2GB or 3GB = user space the rest = OS space Information associated with a process a unique process ID a list of handles an access token holding security information Each process has at least one thread the first thread is created at process creation Win32 API CreateProcess, CreateProcessAsUser, CreateProcessWithLogonW ExitProcess, TerminateProcess, GetExitCodeProcess GetCurrentProcessId, GetEnvironmentStrings

6. No. 6 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes Threads Describes an independent execution within a process Threads form the basis of CPU scheduling Information associated to a thread a state (ready, running, blocked etc.) two stacks for user and kernel execution mode a unique thread ID an access token a context used to save its state a private area for its own local variables There are some kernel threads perform administrative tasks Win32 API CreateThread, CreateRemoteThread ExitThread, GetExitCodeThread, TerminateThread SetThreadPriority, GetThreadPriority, GetCurrentThreadId

7. No. 7 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes Fibers Similar with threads, but scheduled entirely in user space The context switch is not so expensive as with threads – does not need trap to kernel Called lightweight threads Each thread can have multiple fibers The OS is not aware of thread’s fibers Win32 API ConvertThreadToFiber, ConvertFiberToThread CreateFiber, DeleteFiber

8. No. 8 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes Relationship – processes, threads and fibers

9. No. 9 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes Synchronization mechanisms Semaphores – are kernel objects down () = WaitForSingleObject() up () = ReleaseSemaphore() Mutexes (locks) – are kernel objects lock () = WaitForSingleObject() unlock () = ReleaseMutex() Critical sections – similar to mutexes, but local to a process EnterCriticalSection() ExitCriticalSection() Events – are kernel objects manual-reset and auto-reset events WaitForSingleObject() SetEvent(), ResetEvents(), PulseEvent() All of them work on threads, not processes

10. No. 10 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes A list of API Calls

11. No. 11 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes API Calls Examples (1)

12. No. 12 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes API Calls Examples (2)

13. No. 13 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes Scheduling There is no central scheduling thread Windows 2000 is fully preemptive thread switches can occur at any time A thread runs (in kernel mode) the scheduler code when: it blocks on a semaphore, mutex, I/O event etc. it signals an object its quantum expires (usually 20 msec) The scheduler is also called when an I/O operation completes a timed wait expires

14. No. 14 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes Scheduling algorithm (1) Set the process (all threads) priority class SetPriorityClass() the allowed values: real time, high, above normal, normal, below normal, and idle Set the relative priority of a thread within its own process SetThreadPriority() the allowed values: time critical, highest, above normal, normal, below normal, lowest, and idle The system has 32 priorities the 42 possible priority classes are mapped onto the 32 system priories real time (16-31), user (1-15), zero (0), and idle (-1) Works with threads not processes and every thread has associated base priority current priority ( >= base priority )

15. No. 15 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes Scheduling algorithm (2) The highest priority thread is chosen Real-time priorities are fixed User priorities are dynamic A process get a boost when it is woken up because –an I/O operation completes »the amount of boost: 1 for disk, 2 for serial line, 6 for keyboard,8 for sound card etc. –A semaphore is signaled (up) »the amount of boost is 2 When a process consumes its entire quantum, it looses a point from its priority

16. No. 16 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes Mapping of scheduling priorities

17. No. 17 Fall, 2007TUCN. Operating Systems. Lecture 6 Windows processes Scheduling priority classes

18. No. 18 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes The support Processes an instance of a program in execution a collection of data structure fully describes how far the execution has progressed Lightweight processes can share some resources (memory, file descriptors, etc.) Linux support for multithreading Threads many independent execution flows in the same process examples of POSIX-compliant pthread libraries »LinuxThreads »Next GenerationPosix Threading Package (NGPT)

19. No. 19 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Processes description Each process has a unique identifier (pid) returned by getpid() the maximum PID number allowed on Linux is 32,767 A process is created by another process using the fork() system call => parent-child relationship the child process is a copy of the parent, but with its own identity and resources group of processes Processes can communicate or cooperate pipes, signals, shared memory, message queues, semaphores Background processes = daemons

20. No. 20 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Processes information – ps command (1)

21. No. 21 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Processes information – ps command (2)

22. No. 22 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes System calls for processes

23. No. 23 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes The way the shell works

24. No. 24 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Process descriptor (1) A data structure containing all the information related to a process Process state Flags Scheduling information (priority etc.) File descriptors Pointers to the allocated memory areas Each process, even lightweight processes, has its own process descriptor

25. No. 25 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Process descriptor (2)

26. No. 26 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Process state Running currently executed on a CPU or waiting to be executed Interruptible suspended (sleeping) until some conditions become true Uninterruptible similar with the one above, but not responsive to signals Stopped process execution has been stopped Zombie process execution is terminated, but the parent process has not issued a wait() for the terminated child process

27. No. 27 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Process Usage Limits Maximum CPU time signal SIGXCPU sent when limit exceeded Maximum file size allowed signal SIGXFSZ sent when limit exceeded Maximum heap size Maximum number of processes a user can own Maximum number of open files Maximum size of process address space

28. No. 28 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Process creation Traditional way Resources own by the parent are duplicated, and a copy is granted to the child Modern kernels copy-on-write technique clone() system call => lightweight processes »address space »root and working directory »file descriptors table »signal handlers »process identifier (pid) vfork() system call »Processes share the same address space »Parent process is blocked until the child finishes

29. No. 29 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes System calls for threads

30. No. 30 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Threads implementation Based on lightweight processes Thread group –A collection of lightweight processes that share the same pid The fork() semantics only the currently executed thread is activated in the child process atfork() function can be used Signal handling

31. No. 31 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Scheduling policy (1) The set of rules used to determine when and how selecting a new process to run Linux scheduling is based on process preemption time-sharing ranking processes according to their priority The value of processes quantum a good compromise between efficient CPU use and good system responsiveness choose a quantum duration as long as posible, while keeping good system response time

32. No. 32 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Scheduling policy (2) Priorities static – real time processes (between 1 - 99) dynamic – conventional processes »Implicitly favor I/O-bound processes over CPU-bound ones Scheduling classes real-time FIFO real-time Round Robin conventional time-sharing Always chooses the highest priority process to be executed

33. No. 33 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Scheduling algorithm – kernel 2.4 (1) Divide CPU time into epochs an epoch is the time between all runnable processes begin with a new time slice and the time all runnable processes have used up their time slices every process has a specified quantum whose duration is computed when the epoch begins Each process has a base quantum (base priority) the quantum assigned by the scheduler to the process if it has exhausted its quantum in previous epoch about 210 ms can be modified with nice() or setpriority() system calls Dynamic priority of conventional processes base priority + number of ticks of CPU time left to the process before its quantum expires in the current epoch

34. No. 34 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Scheduling algorithm – kernel 2.4 (2) At process creation the number of CPU ticks left to the parent are split in two halves between it and its child Direct invocation The current process must be blocked Lazy invocation quantum expired a process with a greater priority than the current process is woken up sched_setscheduler() or sched_yield() system call is issued

35. No. 35 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Scheduling algorithm – kernel 2.4 (3) At the beginning of a new epoch quantum = quantum/2 + base_priority give preference to I/O bound processes never become larger then 2*(base_priority) How good is a runnable process (goodness) if (process_policy == real_time) process_goodness = 1000 + process_static_priority; if (process_policy == time_sharing && process_quantum==0) process_goodness = 0; if (process_policy == time_sharing && process_quantum>0) if (process was the previous process || share the address space with previous process) process_goodness = process_quantum + process_base_priority + 1; else process_goodness = process_quantum + process_base_priority ;

36. No. 36 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Performance of sched. alg. – kernel 2.4 The algorithm does not scale well –O(n) complexity The predefined quantum is too large for high system loads I/O-bound processes boosting strategy is not optimal Support for real-time application is weak

37. No. 37 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Scheduling algorithm – kernel 2.6 (1) O(1) complexity use a ready queue organized as a stack of priorities queues – called priority array use two priority arrays –Active tasks –have not yet consumed entirely their time slice in the current epoch –Expired – have consumed their time slice in the current epoch a bitmap is used to find quickly the highest priority thread –A bit 1 indicates a non empty priority list in the priority array –O(1) complexity a new time slice for a task is computed when it is inserted in the expired priority array switch between epochs = switch between the two priority arrays –O(1) complexity

38. No. 38 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Scheduling algorithm – kernel 2.6 (2) Range of priorities – 0 ÷ 140 –Real-time (RT) tasks : 0 ÷ 99 –Normal tasks: 100 ÷ 140 Non RT tasks –Average time slice – 100 ms –All tasks have a static priority called nice value (-20 ÷ 19) – never changed by scheduler –Dynamic priority – add to or subtract from static priority Reward I/O tasks Punish CPU-bound tasks Maximum priority bonus is 5 Maximum priority penalty is 5 –Sleep_avg < MAX_SLEEP_AVG add the total sleep time subtract the total runtime

39. No. 39 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Scheduling algorithm – kernel 2.6 (3) Dynamic priority bonus(p) = CURRENT_BONUS(p) - MAX_BONUS / 2; prio(p) = static_prio(p) – bonus(p); CURRENT_BONUS(p) = sleep_avg * MAX_BONUS / MAX_SLEEP_AVG; MAX_BONUS = 10; Time slice –calculated by simply scaling a task’s static priority onto the possible time slice range and making sure a certain minimum and maximum time-slice is enforced time_slice = (MIN_TIMESLICE + ((MAX_TIMESLICE - MIN_TIMESLICE) * (MAX_PRIO - 1 - static_prio) / (MAX_USER_PRIO - 1)))

40. No. 40 Fall, 2007TUCN. Operating Systems. Lecture 6 Linux processes Scheduling related system calls System callDescription nice()Change the priority of a conventional process. getpriority()Get the max. priority of a group of conventional processes. setpriority()Set the priority of a group of conventional processes. sched_getscheduler()Get the scheduling policy of a process. sched_setscheduler()Set the scheduling policy and priority of a process. sched_getparam()Get the scheduling priority of a process. sched_setparam()Set the priority of a process. sched_yield()Relinquish the processor voluntarily without blocking. sched_get_priority_min()Get the minimum priority value for a policy. sched_get_priority_max()Get the maximum priority value for a policy. sched_rr_get_interval()Get the time quantum value for Round Robin policy.

41. No. 41 Fall, 2007TUCN. Operating Systems. Lecture 6 Bibliography [Tann01] Andrew Tannenbaum, “Modern Operating Systems”, second edition, Prentice Hall, 2001, pgs. 690 – 708, pgs. 796 – 809. [BC01] D. Bovet, M. Cesati, “Understanding Linux Kernel”, O’Reilly, 2001, pgs. 65 – 96, pgs. 277 – 298. [JA05] Josh Aas, “Understanding the Linux 2.6 CPU Scheduler”, Silicon Graphics, February 2005.