National Taiwan University OS Project 0 & 1 Advisor: Dr. Chih-Wen Hsueh Student: Tang-Hsun Tu 台灣大學 網媒所 / 資工所 Wireless Networking and Embedded Systems Laboratory Real-Time System Software Group September 7, 2015

National Taiwan University Tang-Hsun Tu /602  Linux Installation  Linux Kernel Compilation  System Call  Project Hints Outline

National Taiwan University Linux Installation

National Taiwan University Tang-Hsun Tu /604  Including shells, libraries, tools, compiler, servers, applications.  Ubuntu, Redhat, Fedora, Mandrake, SuSE, Debian, Ubuntu, Gentoo …  This slides are based on Ubuntu LTS Beta  Linux 3.0  Linux Distribution

National Taiwan University Tang-Hsun Tu /605  Windows can be also installed together.  Modify the boot sequence to boot your computer from CD-ROM.  Make sure your hardware and device.  A clear head and relaxed mind.  Some drinks and food. Before Installation

National Taiwan University Tang-Hsun Tu /606

National Taiwan University Tang-Hsun Tu /607

National Taiwan University Tang-Hsun Tu /608  /dev/hda, /dev/hdb, /dev/hdc, … /dev/hda1, /dev/hda2, …  /dev/sda, /dev/scd0, …  Mount Points / /swap … Disks and Partitions

National Taiwan University Tang-Hsun Tu /609

National Taiwan University Tang-Hsun Tu /6010  Swap partition is usually twice as RAM when it is less than 1GB.  No more than 4 primary partitions including / partition and swap partition.  If the primary partitions are not enough, you can use logical partitions.  Make sure all your mount points are correct. Partition Division

National Taiwan University Tang-Hsun Tu /6011

National Taiwan University Tang-Hsun Tu /6012

National Taiwan University Tang-Hsun Tu /6013  After dividing partitions, you only need to click your mouse.  After installation, reboot and enjoy your Linux!  Installation by text mode is similar to graphic mode. End of Installation

National Taiwan University Tang-Hsun Tu /6014  Install packages in Ubuntu. apt-cache search xxxx apt-get install xxxx  GNU Compiler Collection. gcc –o test test.c./test  Makefile. make  Command-line editor. vim  SSH server and client. ssh –oPort=port Some Useful Commands

National Taiwan University Tang-Hsun Tu /6015  Install ssh server. apt-get install ssh  A ssh client on Windows. Some Useful Tools

National Taiwan University Tang-Hsun Tu /6016  鳥哥的私房菜  Ubuntu 正體中文站  Ubuntu Homepage Reference

National Taiwan University Linux Kernel Compilation

National Taiwan University Tang-Hsun Tu /6018  Kernel is the core of an operating system. Process management Memory management Scheduler Filesystem Virtualization … What is Linux Kernel?

National Taiwan University Tang-Hsun Tu /6019  You can configure your Linux by compiling a new kernel. Add new features, eg. patch kernel. Support new hardware. Disable functions you don’t need. Develop your own kernel.... When Should We Compile Kernel?

National Taiwan University Tang-Hsun Tu /6020  Kernel Website,  You also can download the source codes patched by Ubuntu. Where to D/L Linux Kernel?

National Taiwan University Tang-Hsun Tu /6021  You need to be root to compile kernel. /usr/src/… sudo -i  Download the necessary tools. apt-get update apt-get install kernel-package gcc libncurses5-dev automake gcc libc6-dev build-essential  Go to System/Administration/synaptic package Manager or Get the kernel source codes e.g Prepare Your Kernel Source Code

National Taiwan University Tang-Hsun Tu /6022  Unzip kernel source code. cd /usr/src tar jxvf linux-source-3.X.tar.bz2  You may have many versions of Linux source codes. Prepare Your Kernel Source Code (Cont)

National Taiwan University Tang-Hsun Tu /6023  There are many ways to configure. make config make menuconfig make xconfig...  If you do not know how to configure, you can copy the old config file from /boot. make mrproper cp /boot/config-`uname -r`.config make oldconfig make menuconfig Configure your Kernel

National Taiwan University Tang-Hsun Tu /6024 Configure your Kernel (Cont)

National Taiwan University Tang-Hsun Tu /6025  If you meet some problems (SATA). Device Drivers --->  SCSI device support --->  SCSI device support  SCSI disk support Device Drivers --->  SCSI device support --->  SCSI low-level drivers --->  [*] Serial ATA (SATA) support Configure your Kernel (Cont)

National Taiwan University Tang-Hsun Tu /6026  The first compilation (about 30min ~ 1.5hr): #make clean #make bzImage #make modules  (long time) #make install #make modules_install #mkinitramfs –o /boot/initrd.img-3.x 3.x  do this only if you reinstall modules then reboot!  The second compilation (about 5~10min in our project): #make bzImage #make install then reboot! Kernel Compilation

National Taiwan University Tang-Hsun Tu /6027  See how many cores or hyper-threading on your machine. cat /proc/cpuinfo | grep processor | wc –l e.g. 4  Compile with the number of jobs (threads). make –j4 bzImage make –j4 modules... Speed up Kernel Compilation

National Taiwan University Tang-Hsun Tu /6028  Setup your boot manager. vim /boot/grub/menu.lst  Add the following section (Grub1): title Ubuntu LTS, kernel uuid xxxx kernel /boot/vmlinuz root=UUID=xxxx ro splash initrd /boot/initrd.img Configure your Boot Menu

National Taiwan University Tang-Hsun Tu /6029  Setup your boot manager. vim /boot/grub/grub.cfg  Add the following section (Grub2): Configure your Boot Menu (Cont)

National Taiwan University Tang-Hsun Tu /6030  鳥哥的私房菜  Google Reference

National Taiwan University System Call

National Taiwan University Tang-Hsun Tu /6032  System call is the mechanism used by an application program to request service from the OS.  Users can use it to communicate with kernel.  Here are two approaches developing our own system calls. Using kernel module Modify the source codes of Linux directly Introduction

National Taiwan University Tang-Hsun Tu /6033  Building system calls in kernel module is more flexible than modifying kernel.  When we want to use our system call, just install our kernel modules. Also, if we don’t need it right away, just remove modules. Modifying kernel is not necessary. (But you still need to modify your kernel for OS project 1.) Using Kernel Module

National Taiwan University Tang-Hsun Tu /6034  For sys_call_table, your should extern it in a file such as /arch/x86/kernel/i386_ksyms_32.c. Export sys_call_table extern void* sys_call_table[]; /*variable should be exported. */ EXPORT_SYMBOL(sys_call_table);

National Taiwan University Tang-Hsun Tu /6035  sys_call_table is read-only after kernel version  If you want to try this method using kernel version which is higher than , you have to modify your current kernel source codes and recompile it. Export sys_call_table (Cont)

National Taiwan University Tang-Hsun Tu /6036  Firstly, check your compiled kernel version. uname –a  In x86 32bit vim /usr/src/linux-3.x/arch/x86/kernel/entry_32.S.section.rodata, “a” .section.data, “aw” Export sys_call_table (Cont)

National Taiwan University Tang-Hsun Tu /6037  In x86 64bit vim /usr/src/linux-3.x/arch/x86/kernel/syscall_64.c line 22: delete the “const” Export sys_call_table (Cont)

National Taiwan University Tang-Hsun Tu /6038  Add to export symbol. vim /usr/src/linux-3.x/kernel/kallsyms.c extern void *sys_call_table; EXPORT_SYMBOL(sys_call_table); Export sys_call_table (Cont)

National Taiwan University Tang-Hsun Tu /6039  vim makefile  vim myservice.c Write Your Makefile

National Taiwan University Tang-Hsun Tu /6040  Include and Define.  Extern the “sys_call_table”.  Write your own system call. Write Kernel Module #include /* We're doing kernel work */ #include #define __NR_mysyscall 200 /* define the number of our system call */ typedef void (*sys_call_ptr_t)(void); extern sys_call_ptr_t sys_call_table[]; sys_call_ptr_t orig_sys_call; /* Our system call */ asmlinkage int mysyscall(int arg1) { printk("enter mysyscall()\n"); return arg1 * 10; }

National Taiwan University Tang-Hsun Tu /6041  Initialize the kernel module.  Extern the “sys_call_table”. Write Kernel Module (Cont) /* Initialize the module - replace the system call */ int init_module() { printk("Insert mysyscall module\n"); orig_sys_call = sys_call_table[__NR_mysyscall]; sys_call_table[__NR_mysyscall] = mysyscall; return 0; } /* Cleanup - unregister the appropriate file from /proc */ void cleanup_module() { printk("Remove mysyscall module\n"); sys_call_table[__NR_mysyscall] = orig_sys_call; }

National Taiwan University Tang-Hsun Tu /6042  Compile. make  Insert the module to kernel. insmod./myservice.ko  Remove the module from kernel. rmmod myservice  List the modules in kernel. lsmod Use Kernel Module

National Taiwan University Tang-Hsun Tu /6043  Write an application to use your system call. vim ap.c  Compile and execute. gcc ap.c –o ap./ap 10 User Application #include #define _GNU_SOURCE #include #define __NR_mysyscall 200 int main(int argc,char *argv[]) { printf("%d\n", syscall(__NR_mysyscall, atoi(argv[1]))); return 0; }

National Taiwan University Tang-Hsun Tu /6044  Here are two approaches developing our own system calls. Using kernel module Modify the source codes of Linux directly Build Your Own System Calls

National Taiwan University Tang-Hsun Tu /6045  Create a new file in /usr/src/linux- 3.x/kernel/. vim myservice.c  Add your system call. Write Your System Call #include asmlinkage int sys_myservice(int arg1) { printk("my service is invoked!\n"); return arg1 * 10; }

National Taiwan University Tang-Hsun Tu /6046  In x86 32bit /usr/src/linux-3.x/arch/x86/include/asm/ unistd_32.h  The index must be the last in the list. e.g. #define __NR_myservice 347 Write Your System Call (Cont) - x86_32

National Taiwan University Tang-Hsun Tu /6047  Create an entry (function name) in system call table. /usr/src/linux-3.x/arch/x86/kernel/ syscall_table_32.S.long sys_myservice Write Your System Call (Cont) - x86_32

National Taiwan University Tang-Hsun Tu /6048  In x86 64bit /usr/src/linux-3.x/arch/x86/include/asm/ unistd_64.h  The index must be the last in the list. #define __NR_myservice 309 __SYSCALL(__NR_myservice, sys_myservice) Write Your System Call (Cont) - x86_64

National Taiwan University Tang-Hsun Tu /6049  Define the prototype. /usr/src/linux-3.x/include/linux/syscalls.h asmlinkage int sys_myservice(int arg1); Write Your System Call (Cont)

National Taiwan University Tang-Hsun Tu /6050  Add to makefile to compile. vim /usr/linux-3.x/kernel/Makefile obj-y += myservice.o  Now, you can recompile your kernel. Write Your System Call (Cont)

National Taiwan University Tang-Hsun Tu /6051  Write an application to use your system call. vim ap.c  Compile and execute. gcc ap.c –o ap./ap 10 User Application #include #define _GNU_SOURCE #include #define __NR_myservice 309 int main(int argc,char *argv[]) { printf("%d\n", syscall(__NR_myservice, atoi(argv[1]))); return 0; }

National Taiwan University Tang-Hsun Tu /6052  Socket programming server: socket/bind/listen/accept client: socket/connect  New system calls sendmsg() recvmsg() sendmmsg(), 3.0 ↑ recvmmsg(), ↑ Hints (1/4)

National Taiwan University Tang-Hsun Tu /6053  Since we want measure the number of invoking time of sendmsg()/recvmsg() in sendmmsg()/recvmmsg(), we might need to add counters in /usr/src/Linux- 3.x/net/socket.c. e.g. int nr_called_sendmsg;  If you want to use this variable in your system call or kernel module, you have to export it. EXPORT_SYMBOL(nr_called_sendmsg);  printk can print messages in kernel, use dmesg to check. Hints (2/4)

National Taiwan University Tang-Hsun Tu /6054 Hints (3/4)  A generic interface: syscall(int no,...) __NR_sendmmsg and __NR_recvmmsg syscall() sendmmsg()/recvmmsg()

National Taiwan University Tang-Hsun Tu /6055  Results Hints (4/4) server client 1. syscall number of “recvmmsg()” 2. received messages 3. the invoking times of “recvmsg()” in the kernel 4. time of invoking recvmmsg() 1. typed message 2. syscall number of “sendmmsg()” 3. the invoking times of “sendmsg()” in the kernel 4. time of invoking sendmmsg()

National Taiwan University Tang-Hsun Tu /6056  LXR, Some Useful Tool

National Taiwan University Tang-Hsun Tu /6057  Manpages apt-get install manpages-dev Some Useful Tool (Cont)

National Taiwan University Tang-Hsun Tu /6058  Kernel Website  Socket Programming  LXR  Google  Recvmmsg()  Sendmmsg() Reference

National Taiwan University Tang-Hsun Tu /6059  We set up a desktop PC with virtualization in Lab202. cd /home/lab202/os_cfg Start Xen:./runxen.sh Start Ubuntu VM:./runubu.sh or./runubu2.sh Start WinXP VM:./runxp.sh Start VGA pass-through WinXP VM:  sudo./runvgaxp.sh &  You can compile your kernels on the VMs!  Create your own VM disk. dd or qemu-img-xen create vmdisk.img 8G  Backups are in /home/lab202/backup.  Account and password. Announced in the courses or send a mail to TA. Resource in Lab202

National Taiwan University Tang-Hsun Tu /6060 Q & A