1. Unix Overview CISC2200, Fall 09 1

2. Using Unix/Linux System  Apply for an account  User name and password  Log on and off  through PuTTy, or other telnet/ssh clientPuTTy  Linux server: storm.cis.fordham.edu  After log in, you are in the home directory  associated with each account 2

3. Your first encounter: shell  Graphical user interface vs. command line interface  Shell: interactive command interpreter  On starts up, it displays a prompt character, and waits for user to type in a command line  On input of a command line, shell extracts command name and arguments, searches for the program, and runs it.  When program finishes, shell reads next command line…. 3

4. Linux commands  Command name and arguments:  Some arguments are file names: cp src dest  Some arguments are flags/options: head -20 file  Note that “head 20 file” will print initial 10 lines of file “20”, and file “file”  Wild cards: *, ?, []  rm *.o: remove all.o files  ?: match any one character  [abc]: a or b or c 4

5. Check/Change Login Shell  Many variations: shell, csh, bash, tcsh, ksh  To check the shell you are using  echo $shell  echo $SHELL  echo $0  login shell: default shell for a user, specified in /etc/passwd  To change login shell  chsh 5

6. Some useful tips  Bash stores the commands history  Use UP/DOWN arrow to browse them  Use “history” to show past commands  Repeat a previous command  “! ” or “! (the most recent match)  Search for a command  Type Ctrl-r, and then a string  Bash will search previous commands for a match  File name autocompletion: “tab” key

7. Shell: how does it work  Shell: interactive command interpreter  Start a shell session within another one  Just enter command “bash”  Use ctrl-d or type exit to terminate a session  How does it find the program ?  Environment variable PATH stores a list of paths to search for programs: “set | grep PATH” or “echo $PATH”, “set” to show all variable settings  Builtin commands: history, set, echo, etc.

8. Customize your shell environment  Modify your shell's startup file (in home dir)  sh, ksh:.profile  bash:.profile,.bashrc,.bash_login.bash_profile  csh:.cshrc,.login tcsh:.tcshrc,.login  Note that these all start with dot

9. Set environment variables  Values of environment variables  In sh, ksh, bash: PATH=$HOME/bin:$PATH PS1="You rang? " export PATH PS1 can also do export PS1="Yes? “  In csh, tcsh: setenv PATH $HOME/bin:$PATH set prompt="You rang? "

10. Create customized command shorthand  Aliases  In sh, ksh, bash:  alias ls='ls –F’  alias rm=‘rm –I’: so that you have to confirm the removal  In csh, tcsh  alias ls 'ls –F’

11. File Systems 11

12. File: a sequence of 0 or more bytes containing arbitrary information – Directories are stored as file Hierarchical file system 12 / (root) home staff bi n zhang et c passwd de v cdrom tty24 lib

13. Home directory & Pathname  Absolute pathname, path, specify location of a file or a directory in the complete file structure  /home/staff/zhang is pathname for my home directory  To make life easier:  Working directory (or current directory) concept  To check your current directory: pwd  To change your current directory:  cd  Relative pathname: path names specified relative to current directory 13 “..”: refers to parent dir “.”: current directory “/”: root and seperator in file names “~”: home directory

14. Getting around in the file system  To list files/directories:  ls  To create a subdirectory:  mkdir  To remove a directory:  rmdir 14

15. File manipulating commands  mv: move a file or directory, or rename a file/directory  mv src_path dest_path  cp: copy file or directory  cp –r src_dir dest_dir  rm: remove a file or a directory  rm  rm –r : remove recursively everything under the directory

16. A close look at ls  Simply type “ls” will list names of files under current directory [zhang@storm Demo]$ ls CCodes README SampleCodes ShellScriptes  By default, files are listed in alphabetic order  Files with names starting with “.” is not listed  ls  If is a directory name, list files under the directory

17. Change ls behavior using flags  To list “hidden” files [zhang@storm Demo]$ ls -a... CCodes.HiddenFile README SampleCodes ShellScriptes  To list files in the order of modification time (most recent first) [zhang@storm Demo]$ ls -t README ShellScriptes CCodes SampleCodes

18. Long listing  To get more information about each file [zhang@storm Demo]$ ls -al total 32 drwxr-xr-x 5 zhang staff 4096 2008-01-16 16:01. drwxr-xr-x 41 zhang staff 4096 2008-01-16 16:01.. drwxr-xr-x 2 zhang staff 4096 2008-01-16 15:55 CCodes -rw-r--r-- 1 zhang staff 38 2008-01-16 16:01.HiddenFile -rw-r--r-- 1 zhang staff 53 2008-01-16 15:57 README drwxr-xr-x 2 zhang staff 4096 2008-01-16 15:55 SampleCodes drwxr-xr-x 4 zhang staff 4096 2008-01-16 15:56 ShellScriptes Total disc space taken in blocks (1024 Byte) d means directory Who has permission to read/write the file User name of the owner and its group

19. File permissions  Each file is associated with permission info.  Differentiate three type of users: owner user, user from same group as owner, others  Three type of access  Read (r): use “cat” to open a file to read, use “ls” to list files/directories under a directory  Write (w): modify the contents of the file  Execute (x): run the file, or “cd” to the directory  Trying to snoop into other’s directory [zhang@storm ~]$ ls../roche/ ls: cannot open directory../roche/: Permission denied

20. What’s in a file ?  So far, we learnt that files are organized in a hierarchical directory structure  Each file has a name, resides under a directory, is associated with some admin info (permission, owner)  Contents of file:  Text (ASCII) file (such as your C/C++ source code)  Executable file (commands)  A link to other files, …  To check the type of file: “file ”

21. Display a text file  cat: concatenate input files  more, less: display a file in screen by screen  Go forward using PgDn, Return key  less: can go forward or backward  head, tail: display the first/last 10 lines of a file  head -20 : display first 20 lines

22. Some useful file related utilities  Counting # of lines, words and characters in files  wc  To search files for lines that match a pattern  grep “global warming” articles  grep “traditional medicine” articles  -v option: lines that don’t match the pattern  Where did I define/access a variable named gNumOfOperations ?  grep gNumOfOperations *.[ch]

23. Sort command  Sort the input into alphabetical order line by line  Many options to control sorting order  -r: reverse the normal order  -n: sort in numeric order  -nr: sort in reverse numeric order  +n: sort starting at n+1-th field

24. Compare file contents  Suppose you carefully maintain diff. versions of your projects (so that you can undo some changes), and want to check what’s the difference.  cmp file1 file2: finds the first place where two files differ (in terms of line and character)  diff file1 file2: reports all lines that are different

25. Standard Input/Output 25

26. For each program, three special files are automatically created/opened By default, all three are set to the terminals In C++, cin, cout, cerr In C, extern FILE *stderr, *stdin, *stdout; Standard input/output/error 0 1 2

27. Simple example  A very simple C program #include main() { char yourName[256]; printf ("Your name ?\n"); if (fgets (yourName,256,stdin)==NULL) fprintf (stderr,"No input"); else printf("hello, %s\n", yourName); }

28. Examples  Many Linux prog. reads input from keyboard and writes output to the screen  Command “sort”: read lines from terminal (until Ctrl-D), sorts them and writes to the screen  Very flexible when combined with redirection and pipes 28

29. Redirect input/output/error  Redirect output to a file:  cat tmpfile1 tmpfile2 > newfile  cat tmpfile1 > newfile  cat tmpfile2 >> newfile: append output to the file given  Redirect error output:  cat tmpfile 2>error_out.txt  Redirect input: cat newfile  Note: syntax is different under different shells 29

30. More on redirection  To capture both output and error to same file: ./a.out dd 2> dd : does not work. Error output is not captured. ./a.out dd 2>&1 ./a.out dd >&2  To discard output, redirect it to /dev/null  /dev/null: a special virtual file, “a black hole” ./a.out > /dev/null 2>&1

31. Combining commands together  How many files are there under current directory ? ls > tmp wc –l < tmp rm tmp  Sort current online user by alphabetic order  Is some user login to the system now ? (using grep)

32. Pipe: getting rid of temporary file  Pipe: connect the output of one program to the input of another program  Any prog. that reads from standard input can read from pipe, similarly for the standard output  who am i |./a.out | wc  knows nothing about redirection and pipe

33. Rule of composition  Pipe: one of the fundamental contributions of UNIX system  Design programs to be connected with other programs  Read/write simple, textual, stream-oriented formats  Read from standard input and write to standard output  Filter: program that takes a simple text stream on input and process it into another simple text stream on output

34. Command Pipeline: how ?  Pipe  an inter-process communication mechanism provided by kernel  Has a reading end and a writing end  Any data write to writing end can be read back from the reading end  Read/write pipe is no different from read/write files Reading endWriting end

35. The Power of Pipe  Who is using the most CPU ?  ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10

36. Command Pipeline: how ?*  Shell set things up  create a pipe, “start” two programs simultaneously, with their input/output redirected to the reading/ending end of pipe

37. Process related commands 37

38. The workings of shell*  For each command line, shell creates new child process to run the command  Sequential commands: e.g. date; who  Two commands are run in sequence  Pipelined commands: e.g. ls –l | wc  Two programs are load/execute simultaneously  Shell waits for the completion, and then display prompt to get next command …

39. Run program in background  To start some time-consuming job, and go on to do something else  wc ch * > wc.out &  Shell starts a process to run the command, and does not wait for its completion (i.e., reads and parses next command)  Shell builtin command: wait  Kill a process: kill

40. ps command  To report a snapshot of current processes: ps  By default: report processes belonging to current user and associated with same terminal as invoker.  Example: [zhang@storm ~]$ ps PID TTY TIME CMD 15002 pts/2 00:00:00 bash 15535 pts/2 00:00:00 ps  List all processes: ps -e

41. BSD style output of ps Learn more about the command, using man ps [zhang@storm ~]$ ps axu USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 2112 672 ? Ss Jan17 0:11 init [3] root 2 0.0 0.0 0 0 ? S< Jan17 0:00 [kthreadd] root 3 0.0 0.0 0 0 ? S< Jan17 0:00 [migration/0] root 4 0.0 0.0 0 0 ? S< Jan17 0:00 [ksoftirqd/0] root 5 0.0 0.0 0 0 ? S< Jan17 0:00 [watchdog/0] root 6 0.0 0.0 0 0 ? S< Jan17 0:00 [migration/1] root 7 0.0 0.0 0 0 ? S< Jan17 0:00 [ksoftirqd/1] root 8 0.0 0.0 0 0 ? S< Jan17 0:00 [watchdog/1] root 9 0.0 0.0 0 0 ? S< Jan17 0:00 [migration/2]

42. Some useful commands  To let process keep running even after you log off (no hangup)  Nohup &  Output will be saved in nohup.out  To run your program with low priority  nice &  To start program at specified time (e.g. midnight)  at 2am < file_containing_programs

43. Other useful commands 43

44. Getting help  To check online manual for a command or a library call  man ls, or man fopen  Use PgUp,PgDn, Up Arrow, Down Arrow, Return to move around  GNU’s official documentation format: TexInfo  Use “info ls” for additional description about “ls”

45. Misc. Commands  Send a file to the printer:  lpr  The file should be of format that the printer recognizes, e.g., text file, postscript file (.ps)!  who: who are logged in the system ?  who –a, or who am i  which: show the full path of a command  which bash