1. TDC561 Network Programming Camelia Zlatea, PhD Email: czlatea@cs.depaul.educzlatea@cs.depaul.edu Review UNIX Architecture and Programming

2. Network Programming (TDC561)Page 2 Winter 2003 References  W. Richard Stevens, Network Programming, Vol, I, 2 nd Ed, Prentice Hall PTR, NJ, 1998.  John Shapley Gray, Interprocess Communications in UNIX -- The Nooks and Crannies Prentice Hall PTR, NJ, 1998.  HP-UX man pages http://docs.hp.com/hpux/onlinedocs/B2355-90682/B2355-90682.htmlhttp://docs.hp.com/hpux/onlinedocs/B2355-90682/B2355-90682.html (system calls)

3. Overview UNIX Architecture UNIX processes, threads UNIX program development System calls – fork, exec, etc.

4. Network Programming (TDC561)Page 4 Winter 2003 UNIX Architectural Overview Hardware Architecture Kernel UNIX System Calls Interface Shell’s Executable Programs pipe, filters Commands ApplicationsNetwork ApplicationsDBMSUtilities

5. Network Programming (TDC561)Page 5 Winter 2003 UNIX Features  Portability  Multi-process architecture (multitasking)  Multi-user capability  Ability to initiate asynchronous processes  A hierarchical file system  Device independent I/O operations  User interface: Shell; selectable per user basis

6. Network Programming (TDC561)Page 6 Winter 2003 UNIX Standards  System V Interface Definition (SVID), AT&T  Portable Operating System Interface for Computer Environments (POSIX), based on SVID, IEEE  ANSI C, American National Standard Institute  ANSI/ISO C++ Standard (draft)

7. Network Programming (TDC561)Page 7 Winter 2003 UNIX Implementations  Solaris - Sun Microsystems –SunOS (later called Solaris), Solaris 2.x based on SVR4  HP-UX, Hewlett-Packard, SVR2  Linux, Linus Torvalds, free distribution, PC-based  AIX, IBM, similar to SVR4  IRIX, Silicon Graphics, SVR4

8. Network Programming (TDC561)Page 8 Winter 2003 UNIX Architectural Overview Hardware Architecture Kernel UNIX System Calls Interface Shell’s Executable Programs pipe, filters Commands ApplicationsNetwork Applications DBMSUtilities

9. Network Programming (TDC561)Page 9 Winter 2003 UNIX Features  Portability  Multi-process architecture (multitasking)  Multi-user capability  Ability to initiate asynchronous processes  A hierarchical file system  Device independent I/O operations  User interface: Shell; selectable per user basis

10. Network Programming (TDC561)Page 10 Winter 2003 Multi-process/Multi-user architecture  Virtual Machine –timesharing OS –process, process quantum, process states  Kernel, base OS –manages all HW dependent functions –users have no direct access to it  System Calls Interface –service routine performing user requests

11. Network Programming (TDC561)Page 11 Winter 2003 UNIX Processes  A process may create sub-processes –fork();  A process may terminate –exit();  A process may put itself to sleep temporarily –sleep(20); –pause(); –wait();  Processes –synchronization mechanisms –communication mechanisms

12. Network Programming (TDC561)Page 12 Winter 2003 UNIX Threads  Multiple Processes - concurrency at OS level  Multiple Threads - concurrency at process level –thread = flow of control in a process –multiple threads (stream of instructions) are executed within the same process –threads share code & data (address space) –threads have their own execution stack, PC, register set and states –context switches are avoided –efficient mapping on multi-processor machines

13. Network Programming (TDC561)Page 13 Winter 2003 UNIX Kernel - model Hardware System Call Interface (Library Routines) Process Memory File System I/O Services Mgmt. Mgmt. Scheduler Device Drivers I/O Buffers User Processes Kernel Space User Space

14. Network Programming (TDC561)Page 14 Winter 2003 Apps./Utilities System Calls Hardware Interrupts UNIX KERNEL Hardware

15. Network Programming (TDC561)Page 15 Winter 2003 UNIX Kernel - low level  Process representation, scheduling, dispatching  Memory allocation and de-allocation  Interrupt handling  Low level device control  Disk Mgmt., data buffering  Process synchronization and IPC

16. Network Programming (TDC561)Page 16 Winter 2003 Kernel - services level  Maps user-level requests with device driver actions  A user system call is translated to a call of the kernel routine, providing that requested service  Type of Services: –process creation and termination –I/O services –UNIX file system services –terminal handling services

17. Network Programming (TDC561)Page 17 Winter 2003 Kernel - services level User program: system(“ps -e”); System calls actions: fork(); //process creation service execl(“/bin/ps”, “ps -e”); //code execution

18. Network Programming (TDC561)Page 18 Winter 2003 System Call Interface level  A user mode process is translated into a protected kernel mode process  Now, program can call kernel routines

19. Network Programming (TDC561)Page 19 Winter 2003 User Processes level  User processes running: –shells –Unix commands –utilities –application programs

20. Network Programming (TDC561)Page 20 Winter 2003 UNIX and POSIX API  UNIX API - system calls  UNIX API are called by –C library functions and –C++ standard classes Example: iostream class  Program confirms POSIX.1? #define _POSIX_SOURCE or % CC -D_POSIX_SOURCE *.C

21. Network Programming (TDC561)Page 21 Winter 2003 UNIX and POSIX API  API set to perform: –determine system configuration and user information –file management –process creation and management –inter-process communication –network communication

22. Network Programming (TDC561)Page 22 Winter 2003 UNIX and POSIX API User Process (User Mode of Execution) Kernel mode of execution UNIX API’s level an API is invoked API executed in protected mode API execution completed

23. Network Programming (TDC561)Page 23 Winter 2003 UNIX and POSIX API  Context Switch from user to kernel mode –more overhead than library functions, for the same task –I/O lib.functions are buffered

24. Network Programming (TDC561)Page 24 Winter 2003 Some Unix Terms  TRAP INSTRUCTION –switches a system call from user to kernel mode –handles error conditions –synchronous execution –Ex: division by zero/overflow  INTERRUPT –used by peripherals to request services from the devices handles –asynchronous execution

25. Network Programming (TDC561)Page 25 Winter 2003 Some Unix Terms  SIGNALS software notification of an event  example: –when typing “ctrl-c” an interrupt is generated for the keyboard driver, which notifies the process by sending a signal

26. Network Programming (TDC561)Page 26 Winter 2003 Processes  Process - a program in execution –process - active entity –program - passive entity (binary file)  Address Space - list of memory locations from where a process reads/writes (code/data/stack)  Set of registers (PC, SP,...)  Process Table - linked list of structures associates w/ processes  System Calls - interface between OS and User process

27. Network Programming (TDC561)Page 27 Winter 2003 Process Control Block (process attributes)  Process State –new, ready, running, blocked, terminated  Process Image Map Pointer  Process ID –assigned at process creation  Program Counter (PC) –address of next instruction to be executed  CPU Registers (saved process context)  List of Open File Descriptors  I/O Devices Attached  CPU Scheduling Info (priority)

28. Network Programming (TDC561)Page 28 Winter 2003 Process Image Map Proc. n Proc. 1 Process Image Process Table Text/Code Segment Data Segment Stack Segment Process Control Block

29. Network Programming (TDC561)Page 29 Winter 2003

30. Network Programming (TDC561)Page 30 Winter 2003 Example: /* Display Segment Address Information */ #include extern int etext,edata,end; void main(void) { printf(“etext: %6X\t edata: %6X \t end: %6X \n”, &etext, &edata, &end); }

31. Network Programming (TDC561)Page 31 Winter 2003 Process States  New - process created ( Ex: fork(); )  Ready - process is waiting to be assigned to processor (inserted in ready queue)  Running - instructions are being executed  Blocked - wait for events to occur (inserted in queue)Ex: wait(); pause();  Terminated - normal/abnormal termination (exit();)

32. Network Programming (TDC561)Page 32 Winter 2003 Process Model New Ready Blocked/ Suspended Running User Mode Terminated exit sleep wakeup dispatch created Quantum Expired Running Kernel Mode System Call Interrupt Return Interrupt Interrupt return

33. Network Programming (TDC561)Page 33 Winter 2003  Context of a Process –process state (defined by it’s code) –value of u-area –values of registers the process uses –contents of user and kernel stacks –is associated with process image map  Context Switching –system executes a process in the context of the process –when the kernel decides to execute another process, it does context switching –kernel saves enough information such that it can later switch back to the first process and resumes its execution  Mode Switching –moving from user to kernel mode –kernel save information to return to user mode

34. Network Programming (TDC561)Page 34 Winter 2003  User mode –processes in use mode can access their own instructions and data; NOT kernel or other process’s code or data  Kernel mode –process can access system(kernel) code and data and user addresses –Kernel is part of each process –Kernel executes on behalf of the process P1P2 P3 P4 Kernel Mode User Mode K K U U OS HW

35. Network Programming (TDC561)Page 35 Winter 2003 Context Switching P1 P2 Save state in PCB1 Reload state from PCB2 Save state in PCB2 Reload state from PCB1 OS

36. Network Programming (TDC561)Page 36 Winter 2003  Switching the CPU to another process by saving the state of the old process (PCB) and load the state of the new process (PCB)  Pure Overhead  Performance Bottleneck  Avoid Overhead of Context Switching by introducing new structures: THREADS Context Switching

37. Network Programming (TDC561)Page 37 Winter 2003 Multitasking Sequential Execution Context Switching

38. Network Programming (TDC561)Page 38 Winter 2003 Compilation cc -o file file.c file Man Pages man cc man sys_call man shell_cmd

39. Network Programming (TDC561)Page 39 Winter 2003 UNIX PROCESSES  Process - system support  Fork system call  Process states  Exec system call  Exit function  Background processes  Parent-Child Synchronization (wait)  File Sharing  Examples

40. Network Programming (TDC561)Page 40 Winter 2003 A fork system call: pid_t pid; pid = fork(); if (pid ==-1) {/* fork failure, no more entries in process table*/} else if (pid==0){/*child process*/} else {/* parent process */}

41. Network Programming (TDC561)Page 41 Winter 2003 Fork - System Call  The user process calls fork();  Store system call parameters –arguments, return address, local variables) into the user stack.  Call corresponding kernel service –execution of a trap instruction  In kernel space save: – parameters, return address and local variables for kernel routine  Execute kernel routine  Normal Return –cleans-up kernel stack –switch from kernel to user mode

42. Network Programming (TDC561)Page 42 Winter 2003 Fork - System Call Successful return from system call:  parent and child processes: –share same text segment –identical data segments –identical user & kernel stack segments –same user structure  parent normal return –restores return address from user stack  child “pseudo return” –restores same return address as the parent from its user stack

43. Network Programming (TDC561)Page 43 Winter 2003 Parent Process Child Process File Table I-node Table text data stack text data stack Open Files Current Directory Open Files Current Directory

44. Network Programming (TDC561)Page 44 Winter 2003 Process information #include pid_t getpid(void); /* get current process ID */ pid_t getppid(void); /* get ID of the parent of the process */

45. Network Programming (TDC561)Page 45 Winter 2003 pid_t pid; static int x; x=1; pid = fork(); if (pid < 0 ) {perror(“fork failure”)} else if (pid==0) x++; else x--; printf(“process %d: x=%d\n”, getpid(), x);  What value(s) are printed for variable x?

46. Network Programming (TDC561)Page 46 Winter 2003 #include void main(void) { fork(); printf(“A\n”); fork(); printf(“B\n”); fork(); printf(“C\n”); }  Comment on the above program output.

47. Network Programming (TDC561)Page 47 Winter 2003 #include void main(void) { int i; for (i=1; i<=3; i++) { fork(); printf(“PID=%d i=%d\n”, getpid(), i); } printf(“PID=%d i=%d\n”, getpid(), i); }  Comment on the above program output.

48. Network Programming (TDC561)Page 48 Winter 2003 #include void main(void) { int i; for (i=1; i<=3; i++) { if (fork()==0) break; printf(“PID=%d i=%d\n”, getpid(), i); } printf(“PID=%d i=%d\n”, getpid(), i); }  Comment on the above program output.

49. Network Programming (TDC561)Page 49 Winter 2003 #include void main(void) { int i; for (i=1; i<==3; i++) { if (fork()>0) break; printf(“PID=%d i=%d\n”, getpid(), i); } printf(“PID=%d i=%d\n”, getpid(), i); }  Comment on the above program output.

50. Network Programming (TDC561)Page 50 Winter 2003 Process States  Initiated - fork()  Ready-to-Run –in ready queue for CPU access  Running –process quantum  Blocked –sleep(n); /* deterministic delay */ –pause(); /* non-deterministic delay */  Terminated –exit(int status);  Shell command: %ps [options]

51. Network Programming (TDC561)Page 51 Winter 2003 #include void main(void) { static char buf[2]; if (fork()==0) strcpy(buf,”A\n”); else strcpy(buf,”B\n”); sleep(10); write(1,buf,sizeof(buf)); }  Comment on the above program execution

52. Network Programming (TDC561)Page 52 Winter 2003 Exec - System Call #include int execl(const char *file, const char arg0,.., const argn); int execv(const char *file, char *argv[]);  the process executes the the binary code from file  if failure return -1  if success does not return

53. Network Programming (TDC561)Page 53 Winter 2003  File test1.c #include void main(void) { int i; for (i=1; i<=3;i++) if (fork()==0) {/* process child */ execv(“/bin/ps”,”ps -el”); } else { /* process partent */ printf(“Parent: %d\n”,getpid()); }  %cc -o test1 test1.c  % test1

54. Network Programming (TDC561)Page 54 Winter 2003  File test2.c #include void main(void) { int i; for (i=1; i<=3;i++) if (fork()==0) {/* process child */ execv(“child”,”child”); } else { /* process partent */ printf(“Parent: %d\n”,getpid()); }  %cc -o test2 test2.c

55. Network Programming (TDC561)Page 55 Winter 2003  File child.c #include void main(void) { pid_t pid; printf(“Parent: %d\n”,getpid()); sleep(30); } %cc -o test2 test2.c %cc -o child child.c %test2

56. Network Programming (TDC561)Page 56 Winter 2003 Exit - Function Call #include void exit(int status);  status - parameter returned to parent process  status=0 - normal termination  status#- abnormal termination  Effects: –close all open file descriptors –notifies the parent (by signal) –return status info to parent –if parent no longer exists then PPD:=1(is adopted by process init)

57. Network Programming (TDC561)Page 57 Winter 2003 _exit - System Call #include void _exit(int status);  status - process exit status code  status=0 - normal termination  status#0- abnormal termination  Effects: –terminates a process –closes all open file descriptors –deallocates all process data and stack segment –process becomes a zombie (it is no longer scheduled to run) –init process clean up process table slot

58. Network Programming (TDC561)Page 58 Winter 2003 _exit - System Call #include int main() { cout << "Test for _exit" << endl; _exit(0); return 0; } % CC -o test test.c; test Test for _exit % echo $status 0

59. Network Programming (TDC561)Page 59 Winter 2003 Wait - System Call #include pid_t wait(int *status);  returns -,1 if failure (check errno) –no unwaited-for child process –interrupt signal  returns the child PID, is success  synchronization: –parent waits for a child to terminate –if more child-processes, waits for any

60. Network Programming (TDC561)Page 60 Winter 2003 Waitpid - System Call #include pid_t waitpid(pid_t p, int *status, int opt);  Return value - a child process ID or -1  Argument pid_t p - a child PID -1 waits for any child (same as wait) 0 waits for any child in same group w/ parent >0 waits for child process with this PID=p <-1 waits for any child with GID = |p|

61. Network Programming (TDC561)Page 61 Winter 2003 Waitpid - System Call #include pid_t waitpid(pid_t p, int *status, int opt);  Options values WNOHANG - do not block if status cannot be obtained WNOWAIT - keep process in wait status WUNTRACED - wait any stopped child process

62. Network Programming (TDC561)Page 62 Winter 2003 Parent-Child Process synchronization #include void main(void) { pid_t pid; int i, status; for (i=1; i<=3;i++) if (fork()==0) execv(“child”,”child”); else printf(“Parent: %d\n”,getpid()); while (pid = wait(&status) && pid!=-1) printf(“Child %d is done\n”,getpid()); exit(0); }

63. Network Programming (TDC561)Page 63 Winter 2003 Parent-Child Process synchronization (by priorities) #include void main(void) { pid_t pid[3], p; int i, status; for (i=0; i<=3;i++) if ((pid[i]=fork())==0) execv(“child”,”child”); else printf(“Parent: %d\n”,getpid()); for(i=0;(p=waitpid(pid[i], &status,0)&&p!=-1;i++) if (p!=-1) && (errno==EINTR) printf(“Signal Caught: %s\n”,strerrno(errno)); exit(0); }

64. Network Programming (TDC561)Page 64 Winter 2003 Emulate the C “system” function #include int MySystem( const char *cmd) { pid_tpid; intstatus; switch (pid=fork()) { case -1: return -1; case 0: execl("/bin/sh", "sh", "-c", cmd, 0); perror("execl"); exit(errno); } if (waitpid(pid,&status,0)==pid && WIFEXITED(status)) return WEXITSTATUS(status); return -1;}

65. Network Programming (TDC561)Page 65 Winter 2003 Emulate the C “system” function (cont.) int main() { int rc = 0; charbuf[256]; do { printf("sh> "); fflush(stdout); if (!gets(buf)) break; rc = MySystem(buf); } while (!rc); return (rc); } /* C library function */ intsystem(const char* cmd);

66. Network Programming (TDC561)Page 66 Winter 2003 Processes Hierarchy Pid=0 swapper Pid=1 init Pid gtty Pid sh Pid sh fork - by init exec gtty fork - by init exec gtty exec login exec sh Pid ls -l fork - by sh exec ls

67. Network Programming (TDC561)Page 67 Winter 2003 Shell Processes  Execution of a Shell command –[fork] new process –[exec] command –[wait] shell waits for child process to terminate  % ps -al

68. Network Programming (TDC561)Page 68 Winter 2003 Background Processes  Shell does not wait for child process to complete  Returns prompter immediately % sleep 30 &  shell: [fork] [fork] [exec] –shell child exits immediately after the second [fork], returns prompter –shell grand-child [exec] the command in background –it is adopted by process init