Userland summary Nezer J. Zaidenberg

Today’s topics What have we learned - USERLAND summary Revisited topics execXXX functions POSIX cond Daemons Sync methods we learned EX 2 Pseudo code What have we learned - USERLAND summary Revisited topics execXXX functions POSIX cond Daemons Sync methods we learned EX 2 Pseudo code

Topics we covered - Processes We learned how to create process (fork(2)) execute a run time (execXXX(2)) How to wait(2) and waitpid(2) for process We tried 2 methods of Inter process communications IPC (network and UNIX domain sockets) - THERE ARE MANY MORE We understand process environment - File descriptor table, environment (getenv(2)), heap, stack, global variables Daemon process Signals (sigaction(2)/signal(3)) We learned how to create process (fork(2)) execute a run time (execXXX(2)) How to wait(2) and waitpid(2) for process We tried 2 methods of Inter process communications IPC (network and UNIX domain sockets) - THERE ARE MANY MORE We understand process environment - File descriptor table, environment (getenv(2)), heap, stack, global variables Daemon process Signals (sigaction(2)/signal(3))

Topics we covered - Threads We learned to create POSIX & SDL threads We learned to join threads We learned to sync threads using POSIX & SDL cond POSIX & SDL mutex We know what re-enterant and non-reenterant function is (strtok(3) and strtok_r(3)) We discussed (in lecture) some syncing algorithms We learned to create POSIX & SDL threads We learned to join threads We learned to sync threads using POSIX & SDL cond POSIX & SDL mutex We know what re-enterant and non-reenterant function is (strtok(3) and strtok_r(3)) We discussed (in lecture) some syncing algorithms

Topics we covered - Files We know what file descriptor is (and to dup(2) it) We know how to open(2) and close(2) fd We know how to read(2) write(2) and lseek(2) files We know how to stat(2) to get file info We know how to unlink(2) to delete file or UDS. We know how to mmap(2) munmap(2) and msync(2) files We also know open/scan/readdir(2) API We know what file descriptor is (and to dup(2) it) We know how to open(2) and close(2) fd We know how to read(2) write(2) and lseek(2) files We know how to stat(2) to get file info We know how to unlink(2) to delete file or UDS. We know how to mmap(2) munmap(2) and msync(2) files We also know open/scan/readdir(2) API

Topics we covered - networking Behavior of AF_INET, AF_UNIX SOCK_STREAM, SOCK_DGRAM socket(2), bind(2), listen(2),accept(2), connect(2), send(2), recv(2), close(2) and shutdown(2) sendto(2) and recvfrom(2) select(2) Behavior of AF_INET, AF_UNIX SOCK_STREAM, SOCK_DGRAM socket(2), bind(2), listen(2),accept(2), connect(2), send(2), recv(2), close(2) and shutdown(2) sendto(2) and recvfrom(2) select(2)

Topics we covered UNIX At this point I assume you all know how to Edit a file Compile a project Create make file Run user commands efficiently Read a manpage Get the unix time(2) Hopefully you should also be able to Debug you code Generate core dump and process it Run strace(1) on a running server and see what’s up At this point I assume you all know how to Edit a file Compile a project Create make file Run user commands efficiently Read a manpage Get the unix time(2) Hopefully you should also be able to Debug you code Generate core dump and process it Run strace(1) on a running server and see what’s up

Revisited topics - execXXX Refs 8.9 in APUE 1st edition (p ) 8.10 in APUE 2nd edition (p ) Family of 6 functions (execlp, execvp, execv, execl, execve, execle) execve is the most generic (lowest layer) Those function replace the running image of the current process with new one Refs 8.9 in APUE 1st edition (p ) 8.10 in APUE 2nd edition (p ) Family of 6 functions (execlp, execvp, execv, execl, execve, execle) execve is the most generic (lowest layer) Those function replace the running image of the current process with new one

Exec functions Suffixes The function with l suffix builds argv The functions with the e suffix allow to change environ The functions with the p suffix try each path in PATH environment variable (so if we run ls it will find /bin/ls) Suffixes The function with l suffix builds argv The functions with the e suffix allow to change environ The functions with the p suffix try each path in PATH environment variable (so if we run ls it will find /bin/ls)

Exec functions What does replace the current process image means? The process has it’s memory space (data and program itself) released. Instead a new program is loaded. When the new program terminates the process terminates. There is no way to get back to the old program or to get its data Though it is by no means a must typical use of exec involves a call to fork(2) and wait(2) This is the way your shell (bash(1) or tcsh(1)) calls your programs What does replace the current process image means? The process has it’s memory space (data and program itself) released. Instead a new program is loaded. When the new program terminates the process terminates. There is no way to get back to the old program or to get its data Though it is by no means a must typical use of exec involves a call to fork(2) and wait(2) This is the way your shell (bash(1) or tcsh(1)) calls your programs

System(3) - exec example int my_system (const char *command) { int pid, status; if ((pid = fork()) == -1) return -1; if (pid == 0) { char *argv[4]; argv[0] = "sh"; argv[1] = "-c"; argv[2] = command; argv[3] = 0; execve("/bin/sh", argv, environ); } waitpid(pid,&status,0); return status; } int my_system (const char *command) { int pid, status; if ((pid = fork()) == -1) return -1; if (pid == 0) { char *argv[4]; argv[0] = "sh"; argv[1] = "-c"; argv[2] = command; argv[3] = 0; execve("/bin/sh", argv, environ); } waitpid(pid,&status,0); return status; }

System (cont’d) } do { if (waitpid(pid, &status, 0) == -1) { if (errno != EINTR) return -1; } else return status; } while(1); } // Note that system(3) is not very recommended function and I prefer you will use the above // instead of system(3) in your homework (but using system is legal) // the above code is from the BUGS section of man system. It is suggested as replacement // I suggest you follow the advice } do { if (waitpid(pid, &status, 0) == -1) { if (errno != EINTR) return -1; } else return status; } while(1); } // Note that system(3) is not very recommended function and I prefer you will use the above // instead of system(3) in your homework (but using system is legal) // the above code is from the BUGS section of man system. It is suggested as replacement // I suggest you follow the advice

POSIX cond Ref : APUE 2nd edition (doesn’t appear on first) 11.6 (specifically p ) We know of mutex - I came first therefore it is mine. But lets think of the following scenarios (hint - homework 2) I have a file being received and I need to compress it/uncompress it when receiving is done I have a file being compressed. I have to wake up some threads that will send the file but only after compression is done Ref : APUE 2nd edition (doesn’t appear on first) 11.6 (specifically p ) We know of mutex - I came first therefore it is mine. But lets think of the following scenarios (hint - homework 2) I have a file being received and I need to compress it/uncompress it when receiving is done I have a file being compressed. I have to wake up some threads that will send the file but only after compression is done

Cond Vs. Mutex In Mutex case I am entering the critical section AND prevent other threads from entering there. In cond case I am blocking myself from entering the critical section. I release myself after another thread has finished preparing the critical section for me In Mutex case I am entering the critical section AND prevent other threads from entering there. In cond case I am blocking myself from entering the critical section. I release myself after another thread has finished preparing the critical section for me

Cond example check out p of APUE 2e for a simpler example but their code is not a complete program In our example we will wait for child process to die. (that child process may have done some work for us… such as compress or uncompress file) then we wake up another thread when we are done check out p of APUE 2e for a simpler example but their code is not a complete program In our example we will wait for child process to die. (that child process may have done some work for us… such as compress or uncompress file) then we wake up another thread when we are done

Cond example - singal when child die pthread_cond_t readycond=PTHREAD_COND_INITIALIZER; pthread_mutex_t readymutex=PTHREAD_MUTEX_INIALIZER; void * waitingThread(void * arg) { pthread_mutex_lock(&readymutex); printf (“before waiting for cond\n”); pthread_cond_wait(&readycond, &readymutex ); printf (“after cond was relesaed\n); // do something pthread_mutex_unlock(&readymutex); } pthread_cond_t readycond=PTHREAD_COND_INITIALIZER; pthread_mutex_t readymutex=PTHREAD_MUTEX_INIALIZER; void * waitingThread(void * arg) { pthread_mutex_lock(&readymutex); printf (“before waiting for cond\n”); pthread_cond_wait(&readycond, &readymutex ); printf (“after cond was relesaed\n); // do something pthread_mutex_unlock(&readymutex); }

Cond example 2/2 Int main() { pthread_t tid; pthread_create(&tid,NULL,waitingThread,NULL); my_system(getenv(“program”)); pthread_mutex_lock(&readymutex); pthread_cond_signal(&readycond); pthread_mutex_unlock(&readymutex); } Int main() { pthread_t tid; pthread_create(&tid,NULL,waitingThread,NULL); my_system(getenv(“program”)); pthread_mutex_lock(&readymutex); pthread_cond_signal(&readycond); pthread_mutex_unlock(&readymutex); }

Cond function prototypes #include int pthread_cond_init(pthread_cond_t *cond, const pthread_condattr_t *attr); int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex); int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, const struct timespec *abstime); #include int pthread_cond_init(pthread_cond_t *cond, const pthread_condattr_t *attr); int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex); int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, const struct timespec *abstime);

Cond function prototypes 2/2 #include int pthread_cond_signal(pthread_cond_t *cond); int pthread_cond_broadcast(pthread_cond_t *cond); int pthread_cond_destroy(pthread_cond_t *cond); #include int pthread_cond_signal(pthread_cond_t *cond); int pthread_cond_broadcast(pthread_cond_t *cond); int pthread_cond_destroy(pthread_cond_t *cond);

Initializing conds and mutexes Using macros (PTHREAD_COND_INITIALIZER, PTHREAD_MUTEX_INITIALIZER) Using functions (pthread_mutex_init(2), pthread_cond_init(2)) Unless you know better it is safe to give null for attributes. Cond and mutex attributes are discussed in 12.4 of APUE 2e and refer to things such as is the mutex recursive (i.e. if it is locked twice, by the same thread does it have to be released twice) Most attributes are beyond the scope for us. Using macros (PTHREAD_COND_INITIALIZER, PTHREAD_MUTEX_INITIALIZER) Using functions (pthread_mutex_init(2), pthread_cond_init(2)) Unless you know better it is safe to give null for attributes. Cond and mutex attributes are discussed in 12.4 of APUE 2e and refer to things such as is the mutex recursive (i.e. if it is locked twice, by the same thread does it have to be released twice) Most attributes are beyond the scope for us.

Syncing methods For processes We can use file locking to implement mutex. (this doesn’t work for threads0 We use IPC (for example UDS) to implement cond - We send one byte when we are ready. We select-recv that byte. Waiting in similar way to cond. (this also works between threads) For processes We can use file locking to implement mutex. (this doesn’t work for threads0 We use IPC (for example UDS) to implement cond - We send one byte when we are ready. We select-recv that byte. Waiting in similar way to cond. (this also works between threads) For threads We use mutex to lock critical section We use cond to lock ourselves These mechanisms do not work on Processes!

Daemons We want to run in background detached from any login shell. We fork(2) and kill parent to lose controlling terminal. (controlling terminal belongs to parent) We setsid(2) to detach from login. Then we fork(2) again to lose virtual terminal We want to run in background detached from any login shell. We fork(2) and kill parent to lose controlling terminal. (controlling terminal belongs to parent) We setsid(2) to detach from login. Then we fork(2) again to lose virtual terminal

Additions We usually like to set some signals to our daemon We like to reopen (using dup2) the stdXXX to /dev/null just in case somebody tries to print We like to chdir to (“/”) so that cwd can be removed. (will not be in use) Some daemonize functions do more things We usually like to set some signals to our daemon We like to reopen (using dup2) the stdXXX to /dev/null just in case somebody tries to print We like to chdir to (“/”) so that cwd can be removed. (will not be in use) Some daemonize functions do more things

Syslog Since we closed stdout and stderr we can’t print. Some programmers would like to print anyway… That’s why we use syslog – the UNIX logging facility for all daemons Since we closed stdout and stderr we can’t print. Some programmers would like to print anyway… That’s why we use syslog – the UNIX logging facility for all daemons

Tomer’s way to do ex. 2 What Tomer did was to use ffmpeg generalized reading methods to avoid transferring structs. He defined new protocol and registered it Then all that is left is to implement the protocol methods. What Tomer did was to use ffmpeg generalized reading methods to avoid transferring structs. He defined new protocol and registered it Then all that is left is to implement the protocol methods.

CLIENT IMPLEMENTATION All that the client now need to implement are the methods that are found in osetup.c Tomer has also provided methods that demonstrate using a local file using upex.c The osetup.c should now be overwritten to work with network. – thus you end up with a complete streaming client/server All that the client now need to implement are the methods that are found in osetup.c Tomer has also provided methods that demonstrate using a local file using upex.c The osetup.c should now be overwritten to work with network. – thus you end up with a complete streaming client/server

This to note in Tomer implementation Check main to see that the new protocol is registered. Implement communication protocol anyway you want. Check main to see that the new protocol is registered. Implement communication protocol anyway you want.

Starting next week we study kernel (AT LAST!)