1. 2015. 09. 24. Project 2. “Linux Fundamental” procfs By Dong Jae Shin

2. Computer Engineering Research Laboratory Introduction to Projects  Project 1: Setting up Environment for Project  Project 2: Linux Fundamental  Project 3: System Call and Synchronization  Project 4: File System  Project 5: Device Driver for Embedded H/W  Especially, including “BeagleBoard Development” 2

3. Computer Engineering Research Laboratory Contents  Follow-on Tasks  1. Linux System Administration  2. File System & Software Management  3. Build your own Linux Kernel  Project Tasks  Write a System Monitoring Tools  1. Analyze the given skeleton program (10%)  2. Traverse Process tasklist (20%)  3. per Process Memory Usage (20%)  4. per Process I/O Usage (20%)  5. Process CPU Utilization (10% point Bonus)  Report (30%) ------------------------------------------------------ Max. 100%

4. Computer Engineering Research Laboratory Account Management  Add user  # useradd  Delete user  # userdel  Settings per user  /etc/passwd  Change or create user password  # passwd  # passwd username 4

5. Computer Engineering Research Laboratory File System Management  Add a Virtual Disk  Virtual Machine Settings -> Add -> Hard Disk -> SCSI -> Create a new virtual disk -> 20GB  You need 20GB of free space in your HDD(or SSD) 5

6. Computer Engineering Research Laboratory File System Management  Disk initialization  Format : Create a empty disk with underlying file system  File system  How store and retrieve data in the storage as a logical unit(file and directory)  File system manages your files in their own structure  We’ll learn file system in our lectures later  ex) ext2, ext3, ext4, FAT, NTFS  Commands  mkfs.XXX : format with XXX file system  We will use ext4 6

7. Computer Engineering Research Laboratory File System Management  Format disk we added  # ls /dev/sd*  /dev/sdb is our new disk  # mkfs.ext4 /dev/sdb  Mount new disk  # cd /root && mkdir mnt  # mount /dev/sdb./mnt  Check mounted disk partition  # df -hT  Unmount disk  # umount /dev/sdb  we can also use directory name  # umount /root/mnt 7 # gnome-disks

8. Computer Engineering Research Laboratory Software Install and Uninstall  Ubuntu use “Advanced Packaging Tool”  Install : apt-get install package-name  ex) # apt-get install vim  Uninstall : apt-get remove package-name  Search : apt-cache search search-string  ex) # apt-cache search gcc  Redhat distribution family use “Yellowdog Updater Modified”  # yum install vim  # yum erase vim  # yum search gcc 8

9. Computer Engineering Research Laboratory Kernel  Kernel  Core of Operating System 9 User ProgramUser InterfaceSystem Calls Memory Management Process Management I/O File System NetworkSecurity Device Drivers Hardware User Space Kernel Space

10. Computer Engineering Research Laboratory Linux Distributions  Linux Distributions  Combine & Maintain features 10 Linux Kernel System Software -Compiler -Libraries -User Interface (GUI, CLI) User Program -Office -Database -Browser -Server Program Libraries & Android Runtime Apps -Camera -Browser -SMS Application Framework

11. Computer Engineering Research Laboratory Linux Kernel Download  Download Linux Kernel Source Code  https://www.kernel.org/ https://www.kernel.org/  or http://core.kaist.ac.kr/~EE516/resources/linux- 3.18.21.tar.xzhttp://core.kaist.ac.kr/~EE516/resources/linux- 3.18.21.tar.xz 11

12. Computer Engineering Research Laboratory Compile Linux Kernel  change to root account  # su - root  Install libraries for compile  # sudo apt-get install build-essential libncurses5-dev  Unzip  # cd /root/mnt  # tar -xvf linux-3.18.21.tar.gz  # cd linux-3.18.21  Copy original configuration file  # cp /boot/config-3.19.0-25-generic.config./ 12 Kernel compile needs large disk space about 11GB. We can use new disk we already mounted.

13. Computer Engineering Research Laboratory Compile Linux Kernel  Kernel build configuration  # make menuconfig  just save & exit (we already copy.config file) 13

14. Computer Engineering Research Laboratory Compile Linux Kernel  Linux Kernel Build  Check your # of cores  # cat /proc/cpuinfo |grep processor  Build with multiple processors  if you have 4 processors  # make -j4  It takes long time  Copy compiled kernel & modules  # make modules_install  # make install  Check /boot directory  14

15. Computer Engineering Research Laboratory Booting with new Kernel  # reboot  Keep “shift key down” while rebooting!  Boot menu -> Advanced options -> Select 3.18.21 15  Check your new kernel  # uname -a Compile Time of Kernel

16. Computer Engineering Research Laboratory (1)Kernel Debugging  Print Kernel-level Message  printk  Kernel version of printf  usage) printk(“%s %d %lu”, str, i, j );  Kernel Log Message  tail -f /var/log/kern.log  See also (optional)  for advanced debugging  http://lwn.net/images/pdf/LDD3/ch04.pdf http://lwn.net/images/pdf/LDD3/ch04.pdf 16 procfs

17. Computer Engineering Research Laboratory (2)/proc File System  /proc  a special file system which displays the present of the system  pseudo and virtual file system which resides in the RAM  provides a method of communication between kernel space and user space  Various information provided by proc  https://www.kernel.org/doc/Documentation/filesystems/ proc.txt https://www.kernel.org/doc/Documentation/filesystems/ proc.txt 17 procfs

18. Computer Engineering Research Laboratory (3)/proc File System  CPU Information  cat /proc/cpuinfo 18  Main Memory Information  cat /proc/meminfo Kernel Debugging

19. Computer Engineering Research Laboratory (4)Create a proc entry  Download Skeleton Files  # wget http://core.kaist.ac.kr/~EE516/Projects/Project2/Makefile http://core.kaist.ac.kr/~EE516/Projects/Project2/Makefile  # wget http://core.kaist.ac.kr/~EE516/Projects/Project2/proc_sam ple.c http://core.kaist.ac.kr/~EE516/Projects/Project2/proc_sam ple.c  Compile  # make  Useful ref)  http://linux.die.net/lkmpg/c708.html http://linux.die.net/lkmpg/c708.html 19 procfs

20. Computer Engineering Research Laboratory (5)Test proc entry  Kernel Log  # tail -f /var/log/kern.log  Insert Module  insmod proc_sample.ko  Write Proc File  # echo blahblah > /proc/proc_sample  Read Proc File  # cat /proc/proc_sample  Remove Module  # rmmod proc_sample 20 procfs

21. Computer Engineering Research Laboratory Project Tasks  Make a System Monitoring Tools  System monitoring is an important job especially on the embedded system  Low computing power, Less memory, Limited resources  Monitoring tools  top, ps, netstat, gnome-system-monitor  Tasks  1. Analyze the given skeleton of promon.c.(10%)  2. Traverse Process tasklist (20%)  3. per Process Memory Usage (20%)  4. per Process I/O Usage (20%)  5. Process CPU Utilization (10% point Bonus Task)  Report  (30%) 21 Project Tasks

22. Computer Engineering Research Laboratory Task 1  Analyze the given skeleton of proc_sample.c  Find the functions that you do not know and explain the operations of those functions referring to references and Google. 22

23. Computer Engineering Research Laboratory Task2. Traverse Task List  Every Linux Tasks are managed by doubly linked list 23  Print every task’s PID and Process Name in your proc file system  Textbook Chapter 3. Understanding the Linux Kernel, 3 rd  free e-book version : http://gauss.ececs.uc.edu/Courses/c4022/code/memory/understan ding.pdf http://gauss.ececs.uc.edu/Courses/c4022/code/memory/understan ding.pdf Project Tasks

24. Computer Engineering Research Laboratory task_struct  task_struct (=Process Descriptor) 24  KERNEL/include/linux/sch ed.h  Useful materials : Textbook Chapter 3. of ULK Project Tasks

25. Computer Engineering Research Laboratory task_struct 25 ………

26. Computer Engineering Research Laboratory Task3. Memory Usage  Memory Mapping  RSS : Resident Set Size  VIRT : Virtual Memory Size  Print every task’s VIRT and RSS Memory Size 26 VIRT RSS VIRT Project Tasks Hint) KERNEL/include/linux/mm_types.h struct mm_struct { … unsigned long total_vm; struct mm_rss_stat rss_stat; … }

27. Computer Engineering Research Laboratory Task3. Memory Usage 27 Project Tasks task_struct … *active_mm … *active_mm … mm_struct … total_vm … rss_stat … total_vm … rss_stat include/linux/mm_types.h VIRT Memory rss_stat.count[MM_FILEPAGES] rss_stat.count[MM_ANONPAGES] RSS Memory MM_FILEPAGES : Type of File Mapped Page MM_ANONPAGES : Type of Anonymous Page (Stack, Heap …) include/linux/sched.h

28. Computer Engineering Research Laboratory Task4. Process I/O Stat  I/O Stat is stored in struct task_io_accounting  include/linux/task_io_accounting.h  Hint)  # cat /proc/PID/io  rchar : read bytes by process  wchar : written bytes by process  syscr : # of read system calls  syscw : # of write system calls  read_bytes : read bytes from disk  write_bytes : written bytes to disk 28 write_bytesread_bytes Disk Cache in DRAM Process wcharrchar Project Tasks

29. Computer Engineering Research Laboratory Task4. Process I/O Stat  Hint 1)  fs/proc/base.c  print function of /proc/pid/io  Hint 2)  include/linux/task_io_accounting_ops.h  task_io_accounting_add() function  Hint 3)  task_struct->ioac  task_struct->signal->ioac  for each thread  task_struct->ioac 29 Project Tasks write_bytesread_bytes Disk Cache in DRAM Process wcharrchar

30. Computer Engineering Research Laboratory Task5. CPU Utilization  “top” command  shows the CPU utilization per process 30  Challenging Task (Bonus Point 10%)  Kernel manages CPU ticks consumed by each process in struct task_cputime  in include/linux/sched.h  utime + stime 5. Project Tasks

31. Computer Engineering Research Laboratory Example of Output 31 5. Project Tasks

32. Computer Engineering Research Laboratory Check your results  Check the correctness of your results  Task 2.  # ps -ef  Task 3.  # top -b -n 1  Task 4.  Browser is a good test case (# firefox)  Get pid  ps -ef |grep firefox  # cat /proc/PID/io  Task 5.  # top Project Tasks

33. Computer Engineering Research Laboratory Problems  Problem1.  Describe similarity and difference Process and LWP(Lightweight Process) in the Linux  from the view point of task_struct, PID management, data sharing and scheduling  Problem2.  Describe why we use copy_to_user(), copy_from_user() functions in the proc file system 33

34. Computer Engineering Research Laboratory Submission  Contents  Source Code  Report Project Tasks1~5  Key point of your Source Code  Final Screenshots  Page Limit : about 5 pages for Project Tasks1~5. (except figures)  Submission  Due Date : Oct. 8, PM 23:59  Delay Penalty  10%/day (AM 00:00)  E-mail : djshin@core.kaist.ac.krdjshin@core.kaist.ac.kr  [EE516 Project2] student_number.zip  (various compression format is allowed)