1 File Management Instructors: Fu-Chiung Cheng ( 鄭福炯 ) Associate Professor Computer Science & Engineering Tatung Institute of Technology

2 Outline Unix File System Basic File Manipulation Utilities Advanced File Manipulation Utilities

3 Unix File Systems A file system is a collection of files and directories on a disk or tape in standard UNIX file system format. Low capacity disks (such as floppy disks) usually contain a single file system. Large disks usually are divided into several regions (partitions), each containing a file system. Each UNIX file system contains four major parts: A. boot block: loader programs for booting B. superblock: key information of the file system C. i-node table: i-nodes store information about files. D. data block: file storage

4 File System layout Block 0: bootstrap Block 1: superblock Block 2 Block n... Block n+1 The last Block... Block 2 - n:i-nodes Block n+1 - last:Files

5 Boot Block A boot block may contains several physical blocks. Note that a physical block contains 512 bytes (or 1K or 2KB) A boot block contains a short loader program for booting It is blank on other file systems.

6 Super Block Superblock contains key information about a file system Superblock information: A. Size of a file system and status: label: name of this file system size: the number of logic blocks date: the last modification date of super block. B. information of i-nodes the number of i-nodes the number of free i-nodes C. information of data block: free data blocks. The information of a superblock is loaded into memory.

7 I-nodes i-node: index node (information node) i-list: the list of i-nodes i-number: the index of i-list. The size of an i-node: 64 bytes. i-node 0 is reserved. i-node 1 is the root directory. i-node structure: next page

8 I-node structure mode owner timestamp Size Block count Direct blocks 0-9 Double indirect Triple indirect Single indirect Data block Indirect block... Data block... Indirect block... Reference count

9 I-node structure mode: A. type: file, directory, pipe, symbolic link B. Access: read/write/execute (owner, group,) owner: who own this I-node (file, directory,...) timestamp: creation, modification, access time size: the number of bytes block count: the number of data blocks direct blocks: pointers to the data single indirect: pointer to a data block which pointers to the data blocks (128 data blocks). Double indirect: (128*128=16384 data blocks) Triple indirect: (128*128*128 data blocks)

10 Data Block A data block has 512 bytes. A. Some FS has 1K or 2k bytes per blocks. B. See blocks size effect (next page) A data block may contains data of files or data of a directory. File: a stream of bytes. Directory format: i-#NextsizeFile namepad

11 Report.txt home john bin find alexjenny notes grep i-#Next10Report.txtpadi-#Next3 binpadi-#Next5notespad0Next

12 Block size

13 Current Working Directory login: kc password:****** Welcome to UNIX ! $ pwd /home/kc $ cd /usr/kc/source $ pwd /usr/kc/source pwd: print current working directory Report.txt home kc source find alex notes grep

14 Changing Working Directory The cd utility makes another directory the working directory. $ cd /home/alex/literature $ pwd /home/alex/literature $ cd $ pwd /home/kc

15 List Files list the contents of directories Example: list the current working direcory $ ls Report.txtsource notes

16 Current directory Per-process data region Boot Block... SuperBlock i-node... Current Dir Report.txt source notes... I-nodes Data Blocks

17 List Files list with options -l: long format $ ls -l -rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txt drwxrwxr-x 2 kc pubs 512 May 5 14:03 source drwxr-xr-x 1 jo staff 512 Jun notes permissions number of links ownergroup user Size Last access time code File or directory name

18 File Code (types)

19 Remove Files rm fileName: remove fileName in current directory rm * : remove all files in current directory rm -rf dirName: remove all the files in dirName rm -i fileName: interactive option $ ls -l -rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txt drwxrwxr-x 2 kc pubs 512 May 5 14:03 source drwxr-xr-x 1 jo staff 512 Jun notes $rm * $ls -l total 0 $

20 Move Files mv file or files to someOtherDir $ ls -l -rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txt drwxrwxr-x 2 kc pubs 512 May 5 14:03 source drwxr-xr-x 1 jo staff 512 Jun notes $mv *../notes $ls -l total 0 $ls -l../notes -rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txt drwxrwxr-x 2 kc pubs 512 May 5 14:03 source drwxr-xr-x 1 jo staff 512 Jun notes

21 Copy Files Copy file will duplicate files $ ls -l -rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txt drwxrwxr-x 2 kc pubs 512 May 5 14:03 source drwxr-xr-x 1 jo staff 512 Jun notes $ls -l../notes total 0 $cp *../notes $ls -l../notes -rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txt drwxrwxr-x 2 kc pubs 512 May 5 14:03 source drwxr-xr-x 1 jo staff 512 Jun notes

22 Links link: a reference to a file or directory two kinds of link: hard links and soft link hard link: ln file1 file2 (create a hard link file2 to file1) soft link: ln -s file1 file2 (create a soft link file2 to file1)

23 Hard Links(I) $ ls -l -rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txt drwxrwxr-x 2 kc pubs 512 May 5 14:03 source drwxr-xr-x 1 jo staff 512 Jun notes $ln Report.txt hardLink.txt $ls -l -rw-r--r-- 2 kc pubs 3355 May 2 10:52 Report.txt -rw-r--r-- 2 kc pubs 3355 May 9 7:22 hardLink.txt drwxrwxr-x 2 kc pubs 512 May 5 14:03 source drwxr-xr-x 1 jo staff 512 Jun notes $rm Report.txt $ls -l -rw-r--r-- 1 kc pubs 3355 May 9 7:22 hardLink.txt drwxrwxr-x 2 kc pubs 512 May 5 14:03 source drwxr-xr-x 1 jo staff 512 Jun notes $more hardLink.txt the content is shown here

24 Hard Links (II) i-#Next10Report.txtpad i-#Next12hardLink.txtpad Reference count=2 Description of file i-node File

25 Remove Hard Links (III) i-#Next12hardLink.txtpad Reference count=1 Description of file i-node File

26 Soft Links (I) $ ls -l -rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txt drwxrwxr-x 2 kc pubs 512 May 5 14:03 source drwxr-xr-x 1 jo staff 512 Jun notes $ln -s Report.txt softLink.txt $ls -l -rw-r--r-- 1 kc pubs 3355 May 2 10:52 Report.txt lrwxrwxrwx 1 kc pubs 3355 May 9 7:22 softLink.txt drwxrwxr-x 2 kc pubs 512 May 5 14:03 source drwxr-xr-x 1 jo staff 512 Jun notes $rm Report.txt $ls -l lrwxrwxrwx 1 kc pubs 3355 May 9 7:22 softLink.txt drwxrwxr-x 2 kc pubs 512 May 5 14:03 source drwxr-xr-x 1 jo staff 512 Jun notes $more softLink.txt softLink.txt: No such file or directory $

27 Soft Links (Symbolic) II i-#Next10Report.txtpad i-#Next12softLink.txtpad Reference count=1 i-node File Reference count=1 Description of file i-node /home/kc /Report.txt

28 Soft Links (Symbolic) III 0Next10Report.txtpad i-#Next12softLink.txtpad Reference count=1 i-node /home/kc /Report.txt

29 Change File modes Permission type: r: read w:write x:execute Permission object: user(u), Group(g), others(o), all(ugo) chmod: change permission + add permission = set permission - remove permission Example: chmod -R 755 SomeDir $ chmod a+rw Report.txt $ ls -l Report.txt -rw-rw-rw- 1 kc pubs 3355 May 2 10:52 Report.txt

30 Change Owner and Group change owner: you need to have the right (such as superuer) to do so $ chown john Report.txt $ ls -l Report.txt -rw-rw-rw- 1 john pubs 3355 May 2 10:52 Report.txt $ chgrp staff Report.txt $ ls -l Report.txt -rw-rw-rw- 1 john staff 3355 May 3 11:45 Report.txt

31 Create & Remove Directories mkdir: create a new directory rmdir: remove a directory Report.txt home kc source find alex notes grep $ mkdir /home/kc/notes/UNIX $ ls /home/kc/notes UNIX $rmdir /home/kc/notes/UNIX … $rm -r /home/kc/source UNIX

32 Search Files/directories / home john templiterature promo demo $find /home -name demo -print /home/alex/demo /home/john/demo alex demo find: locate the misplaced files (or directories)

33 Search Files/directories $ find /usr -name ‘v*[0-9]’ -print /usr/share/lib/terminfo/v/vt100 /usr/share/lib/terminfo/v/vt52 …. locate files/directories that start with a v and end with a digit

34 Search Files/directories $ find /usr -size print /usr/lib/libc.so /usr/lib/libnsl.so …. locate files whose size have more than 1000 blocks

35 Search Files/directories $ find. -mtime -1 -print./notes/softLink.txt./notes/hardLink.txt …. locate files that have been modified within the last day (24 hrs)

36 Search Files/directories $ find / -name core -exec rm {} \; $ find / -name core -ok rm {} \; Lab: $ mkdir d1 d2 d3 $ cp anyFile d1/xxx $ cp anyFile d2/xxx $ cp anyFile d3/xxx $ find. -name xxx -ok rm {} \; locate files and then remove them

37 Compression Two lossless compression: compress and pack compressed files: *.z $ ls -l -r-xr-xr-x 1 kc other Feb 4 10:52 ls -r-xr-xr-x 1 kc other Feb 4 10:52 vi -r-xr-xr-x 1 kc other Feb 4 10:53 who $ pack * UX:pack: INFO: ls: 24.7% Compression UX:pack: INFO: vi: % Compression UX:pack: INFO: who: 19.6% Compression $ ls -l -r-xr-xr-x 1 kc other Feb 4 10:52 ls.z -r-xr-xr-x 1 kc other Feb 4 10:52 vi.z -r-xr-xr-x 1 kc other Feb 4 10:53 who.z

38 Compression Two lossless compression: compress and pack compressed files: *.Z $ ls -l -r-xr-xr-x 1 kc other Feb 4 10:52 ls -r-xr-xr-x 1 kc other Feb 4 10:52 vi -r-xr-xr-x 1 kc other Feb 4 10:53 who $ compress -v * ls: Compression 34.22% -- replaced with ls.Z vi: Compression 34.43% -- replaced with vi.Z who: Compression 30.90% -- replaced with who.Z $ ls -l -r-xr-xr-x 1 kc other Feb 4 10:54 ls.Z -r-xr-xr-x 1 kc other Feb 4 10:54 vi.Z -r-xr-xr-x 1 kc other Feb 4 10:55 who.Z

39 Compression Restore compressed files(*.z): unpack Restore compressed files(*.Z): uncompress $ unpack * $ ls -l -r-xr-xr-x 1 kc other Feb 4 10:52 ls -r-xr-xr-x 1 kc other Feb 4 10:52 vi -r-xr-xr-x 1 kc other Feb 4 10:53 who $ uncompress * $ ls -l -r-xr-xr-x 1 kc other Feb 4 10:54 ls -r-xr-xr-x 1 kc other Feb 4 10:54 vi -r-xr-xr-x 1 kc other Feb 4 10:55 who

40 Collect Files tar(tape archive): collect files into a single file Space are allocated in clusters of two, four or even eight blocks at a time. (block size = 512 bytes) Large disks usually have large cluster size. If you create a file containing just a single character, Unix system will typically allocate 2 or byte blocks Thus compressing a 500-byte file into 256 bytes won’t save any space. Collecting (tar) small files into a single file will save space.

41 Collect Files create a tar file: -c $ ls -l * -r-xr-xr-x 1 kc other Feb 4 10:52 don.ltr -r-xr-xr-x 1 kc other Feb 4 10:52 don2.ltr -r-xr-xr-x 1 kc other Feb 4 10:53 jane.ltr $ tar -cvf aTarFile.tar *ltr a don.ltr 37 tape blocks a don2.ltr 292 tape blocks a jane.ltr 129 tape blocks $ ls -l * -r-xr-xr-x 1 kc other Feb 4 10:52 don.ltr -r-xr-xr-x 1 kc other Feb 4 10:52 don2.ltr -r-xr-xr-x 1 kc other Feb 4 10:53 jane.ltr -r-xr-xr-x 1 kc other Feb 4 10:55 aTarFile.tar

42 Collect Files list the content of a tar file: -t $ ls -l * -r-xr-xr-x 1 kc other Feb 4 10:52 don.ltr -r-xr-xr-x 1 kc other Feb 4 10:52 don2.ltr -r-xr-xr-x 1 kc other Feb 4 10:53 jane.ltr -r-xr-xr-x 1 kc other Feb 4 10:55 aTarFile.tar $ tar -tvf aTarFile.tar -r-xr-xr-x 1 kc other Feb 4 10:52 don.ltr -r-xr-xr-x 1 kc other Feb 4 10:52 don2.ltr -r-xr-xr-x 1 kc other Feb 4 10:53 jane.ltr $ rm *.ltr

43 Collect Files extract a tar file: -x $ tar -xvf aTarFile.tar $ ls -l * -r-xr-xr-x 1 kc other Feb 4 10:52 don.ltr -r-xr-xr-x 1 kc other Feb 4 10:52 don2.ltr -r-xr-xr-x 1 kc other Feb 4 10:53 jane.ltr -r-xr-xr-x 1 kc other Feb 4 10:55 aTarFile.tar

44 Examine File Type file command determines the type information of files. Type information: directory, binary file, ascii file $ ls Readmemakefilemeggaa.czsrc $ file * Readme:ascii text makefile:ascii text meggaa.cc program text zsrc:directory

45 Examine Disk Usage du command show how much disk storage (blocks) of your files. $ ls -l * -r-xr-xr-x 1 kc other Feb 4 10:52 don.ltr -r-xr-xr-x 1 kc other Feb 4 10:52 don2.ltr -r-xr-xr-x 1 kc other Feb 4 10:53 jane.ltr $ du 37./don.ltr 292./don2.ltr 129./ jane.ltr $ du /ect/fc … $ du -s /ect/fc 2750/ect/fs

46 Examine Free Disk Space df command show how much free disk storage (blocks) some system you may use -k option to show K bytes. $ df -k Filesystem kbytes used avail cap mount /dev/dsk/c0t3d03s % / /dev/dsk/c0t3d03s % /homes swap % /tmp $ df -k. Filesystem kbytes used avail cap /dev/dsk/c0t3d03s % /home2 $ df. /home2 (/dev/dsk/c0t3d03s2 ): blocks files

47 Dump Files od (octal dump) command show a file with octal, decimal, ASCII, hexadecimal format. $ cat spices thymenutmeg sagecumin saltpepper $ od -c spices t h y m e \t n u t m e g \n s a g e \t c u m i n \n s a l t \tp e p p e r \n

48 On-line Manual Always use on-line manual (man) to find detail syntax and options $ man cat... $ man od...

49 End of File Management Lecture