1. Real-time Systems Lab, Computer Science and Engineering, ASU Linux Modules and Device Drivers (ESP – Fall 2014) Computer Science & Engineering Department Arizona State University Tempe, AZ 85287 Dr. Yann-Hang Lee yhlee@asu.edu (480) 727-7507

2. Real-time Systems Lab, Computer Science and Engineering, ASU Software Development Environment  IDE – Eclipse, KDevelop  Linux and GNU tools  GCC, GDB, and GUI-based tools  applications in user space  operate device interface in user space (read/write of memory-mapped I/Os)  interrupt service routines in kernel space  kernel modules and memory copy between user and kernel space 1

3. Real-time Systems Lab, Computer Science and Engineering, ASU Development Environment of Linux Host PC workstation gcc cross- compiler Linux Eclipse IDE GDB Server Target board Board support package Applications Linux or Windows GDB debugger 2

4. Real-time Systems Lab, Computer Science and Engineering, ASU What is “kernel”  The basic component of an operating system  to provide the lowest-level abstraction layer for the resources (especially processors and I/O devices).  available to application processes through inter-process communication mechanisms and system calls  Kernel space and user space  What are system calls  Which are systems calls – cc, make, ls, cat, grep, read, open, printf, malloc, etc.  How can we set the baud rate of a serial port?  Configuration of a hyperterminal  stty -F /dev/ttyS2 ospeed 38400  Ioctl to get and set terminal attributes (defined in struct termios)  The mechanism of making system calls and passing parameters 3

5. Real-time Systems Lab, Computer Science and Engineering, ASU Kernel Components  Process Management  Process life cycle, Inter Process Communication, I/O  Scheduling (long-term, short-term)  Memory Management  Virtual memory, management, security  File System  File system tree, management, security  Device Control  Almost every system operation maps to a physical device  The code used to do those operations is called Device Driver  Networking  Collect incoming packets and De-Multiplexing them  Deliver outgoing packets 4

6. Real-time Systems Lab, Computer Science and Engineering, ASU Memory Spaces (1)  The logical address space of a process is divided into two parts  – 0x00000000 to PAGE_OFFSET-1 can be addressed in either user or kernel mode  – PAGE_OFFSET to 0xffffffff can be addressed only in kernel mode  – PAGE_OFFSET is usually 0xc00000000  Peripheral devices are controlled by writing and reading their registers. Where are these registers?  I/O Port: request an IO port region and inb (outb) from  I/O Memory:  Request a memory region  Devices at physical addresses which have to be mapped to virtual addresses for software to access. 5

7. Real-time Systems Lab, Computer Science and Engineering, ASU Memory Spaces (2)  Physical memory  Virtual memory (user and kernel space)  Linux ARM -- Linux/Documentation/arm/memory.txt (X86 virtual memory layout) 6

8. Real-time Systems Lab, Computer Science and Engineering, ASU Linux Kernel Modules  Modules can be compiled and dynamically linked into kernel address space.  Useful for device drivers that need not always be resident until needed. (why?)  Extend the functionality of the kernel without rebuilding and rebooting the system.  Kernel keeps a symbol table  Symbols accessible to kernel-loadable modules appear in /proc/kallsyms.  EXPORT_SYMBOL(), which exports to any loadable module, or  EXPORT_SYMBOL_GPL(), which exports only to GPL-licensed modules.  Can call any functions exported by the kernel  no library is linked to modules 7

9. Real-time Systems Lab, Computer Science and Engineering, ASU Kernel Modules  Pieces of code that can be loaded and unloaded into the kernel.  a module registers itself in order to serve future requests  Is a part of kernel – printf or printk  An example module #include // Needed by all modules #include // Needed for KERN_ALERT #include // Needed for the macros static int hello_2_init(void) { printk(KERN_ALERT "Hello, world 2\n"); return 0; } static void hello_2_exit(void) { printk(KERN_ALERT "Goodbye, world 2\n"); } module_init(hello_2_init); module_exit(hello_2_exit); 8

10. Real-time Systems Lab, Computer Science and Engineering, ASU HelloWorld Kernel Module Example (1) /* Hello World kernel module program */ #include static int hello_init(void) { int i; struct timeval curTime;// keeps the time since the Epoch struct tm bkd_time;// containing a calendar date and time for (i=0; i<10; i++) { do_gettimeofday(&curTime); time_to_tm(curTime.tv_sec, 0, &bkd_time); printk(KERN_ALERT "Hello, world. The current time is: %d:%d:%d:%ld\n", bkd_time.tm_hour, bkd_time.tm_min, bkd_time.tm_sec, curTime.tv_usec); msleep(i*1000); } } static void hello_exit(void) { printk(KERN_ALERT "Goodbye, cruel world\n"); } module_init(hello_init); module_exit(hello_exit); 9

11. Real-time Systems Lab, Computer Science and Engineering, ASU HelloWorld Kernel Module Example (2) # Makefile for Hello World kernel module CC = i586-poky-linux-gcc ARCH = x86 CROSS_COMPILE = i586-poky-linux- SROOT=/opt/clanton-full/1.4.2/sysroots/i586-poky-linux/ obj-m = HelloModule.o all: make ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE) -C $(SROOT)/usr/src/kernel M=$(PWD) modules ---- copy to the target scp HelloModule.ko root@10.218.101.25:/home/.......root@10.218.101.25:/home/ ---- install and remove the module insmod HelloModule.ko rmmod HelloModule 10

12. Real-time Systems Lab, Computer Science and Engineering, ASU Linking and Unlinking Modules 11

13. Real-time Systems Lab, Computer Science and Engineering, ASU Programs for Linking and Unlinking Modules (1)  insmod  Reads from the name of the module to be linked  Locates the file containing the module's object code  Computes the size of the memory area needed to store the module code, its name, and the module object  Invokes the create_module( ) system call  Invokes the query_module( ) system call  Using the kernel symbol table, the module symbol tables, and the address returned by the create_module( ) system call, relocates the object code included in the module's file.  Allocates a memory area in the User Mode address space and loads with a copy of the module object  Invokes the init_module( ) system call, passing to it the address of the User Mode memory area  Releases the User Mode memory area and terminates 12

14. Real-time Systems Lab, Computer Science and Engineering, ASU Programs for Linking and Unlinking Modules (2)  lsmod  reads /proc/modules  rmmod  From reads the name of the module to be unlinked.  Invokes the query_module( )  Invokes the delete_module( ) system call, with the QM_REFS subcommand several times, to retrieve dependency information on the linked modules.  modprobe  loads a module into the kernel.  check any module dependency and load any other modules that are required – stacking of modules 13

15. Real-time Systems Lab, Computer Science and Engineering, ASU Module Implementation  The kernel considers only modules that have been loaded into RAM by the insmod program and for each of them allocates memory area containing:  a module object  null terminated string that represents module's name  the code that implements the functions of the module  Building  Linux kbuild  make -C ~/kernel-2.6 M=`pwd` modules to build modules.ko  “M=“ is recognized and kbuild is used  ~/kernel-2.6 is the kernel source directory  pwd ?? 14

16. Real-time Systems Lab, Computer Science and Engineering, ASU Device Driver Basics  Purpose:  a well defined and consistent interface to handle requests for device operations  isolate device-specific code in the drivers  A software interface to hardware devices  resides in kernel or user spaces  Classification  character device (terminal)  block (disk) -- with buffer cache  network  pseudodevice  When to call a device driver  configuration, I/O operations, and interrupt OS specific code I/O class specific code Hardware specific code I/O adapters device driver 15

17. Real-time Systems Lab, Computer Science and Engineering, ASU Char and Block Devices  Character Devices  Accessed as a stream of bytes (like a file)  Typically just data channels or data areas, which allow only sequential access  Char devices are accessed by means of filesystem nodes  Example: /dev/tty1 and /dev/lp0  Driver needs to implement at least the open, close, read, and write system calls  Block Devices  provides access to devices that transfer randomly accessible data in fixed- size blocks  Examples?  Design Challenges  Concurrency, performance, and portability 16

18. Real-time Systems Lab, Computer Science and Engineering, ASU Char Device  cdev -- the kernel internal structure to represents a character device file  Defined in struct cdev { struct kobject kobj; struct module *owner; struct file_operations *ops; struct list_head list; dev_t dev; unsigned int count; }; struct file_operations ldd_fops = {.owner = THIS_MODULE,.llseek = ldd_llseek,.read = ldd_read,.write = ldd_write,.ioctl = ldd_ioctl,.open = ldd_open,.release = ldd_release }; 17

19. Real-time Systems Lab, Computer Science and Engineering, ASU Driver Interface for Character Devices struct file_operations { // defined in inlcude/linux/fs.h struct module *owner; loff_t (*llseek) (struct file *, loff_t, int); ssize_t (*read) (struct file *, char __user *, size_t, loff_t *); ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *); ssize_t (*aio_read) (struct kiocb *, const struct iovec *, unsigned long, loff_t); ssize_t (*aio_write) (struct kiocb *, const struct iovec *, unsigned long, loff_t); long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long); long (*compat_ioctl) (struct file *, unsigned int, unsigned long); int (*mmap) (struct file *, struct vm_area_struct *); int (*open) (struct inode *, struct file *); int (*flush) (struct file *, fl_owner_t id); int (*release) (struct inode *, struct file *); int (*fsync) (struct file *, int datasync); int (*fasync) (int, struct file *, int); int (*lock) (struct file *, int, struct file_lock *); ………………… }; 18

20. Real-time Systems Lab, Computer Science and Engineering, ASU User Program & Kernel Interface 19

21. Real-time Systems Lab, Computer Science and Engineering, ASU Basic Device Driver Structure  Major number to identify the driver associated with the device  Minor number provides a way for the driver to differentiate multiple devices.  dev_t type in : 8bis for major and 8 bits for minor in kernel 2.4, 12bis for major and 20 bits for minor in kernel 2.6. “/tty0/” 1 “/xx1/” 3 “/pty0/” 1 Device- dependent data Device list (of device descriptors) Driver table (function pointers) File descriptor table 01230123 20

22. Real-time Systems Lab, Computer Science and Engineering, ASU Register Char Device (1) static dev_t myDev = 0; static char myName[] = "TestDevice"; static int result = 0; static int hello_init(void) { if((result = alloc_chrdev_region(&myDev,0,1,myName))!= 0){ printk("Unable to allocate device number\n"); return -1; } printk(KERN_ALERT "My major is %d, minor is %d\n“, MAJOR(myDev),MINOR(myDev)); return 0; } static void hello_exit(void) { if(result == 0){ unregister_chrdev_region(myDev, 1); } 21

23. Real-time Systems Lab, Computer Science and Engineering, ASU Register Char Device (2)  Device numbers registration and release  Manually-assigned registration  Dynamically-allocated registration  Release int register_chrdev_region( dev_t first,# The beginning device number unsigned int count,# Number of continuous device number char *name# Device name (shown in /proc/devices) ); int alloc_chrdev_region( dev_t *dev,# output-only that holds the first device unsigned int firstminor,# The beginning of first minor number unsigned int count, char *name ); void unregister_chrdev_region( dev_first,# The first device number going to release unsigned int count ); 22

24. Real-time Systems Lab, Computer Science and Engineering, ASU Device Registration  Allocate and initialize a cdev struct  cdev_alloc or cdev_init  cdev gets initilized with a file_opreations structure  cdev_add to register operatons of your device driver  int cdev_add(struct cdev *dev, dev_t num, unsigned int count)  cdev_del — remove a cdev from the system 23

25. Real-time Systems Lab, Computer Science and Engineering, ASU A Memory Buffer Driver Module Example  As a module,  __init gmem_init(void)  __exit gmem_exit(void)  In gmem_init, create and register a character device with file operations  open, release, read, and write  device file is then created in file system  User program can use device name as a file and invoke file operations on the device  inode  cdev  gmem_dev (a super-class of cdev containging device related data)  file 24 q (Add a code example)

26. Real-time Systems Lab, Computer Science and Engineering, ASU Open System Call  The open system maps onto an open operation registered by a driver module  The kernel first does some processing (e.g., finds the inode struct and file struct for the driver  Invokes the open operation  int open(const char *pathname, int flags);  int open(const char *pathname, int flags, mode_t mode); // return the new file descriptor,  int (*open) (struct inode *, struct file *); 25

27. Real-time Systems Lab, Computer Science and Engineering, ASU File Struct  The file structure represents an open file.  a kernel structure  created by the kernel on open and is passed to any function that operates on the file  defined in  mode_t f_mode : readable or writable  loff_t f_pos: current reading or writing position.  struct file_operations *f_op  void *private_data struct file { struct list_head f_list; struct dentry *f_dentry; struct vfsmount *f_vfsmnt; struct file_operations *f_op; atomic_t f_count; unsigned int f_flags; mode_t f_mode; loff_t f_pos; unsigned long f_reada, f_ramax, f_raend, f_ralen, f_rawin; struct fown_struct f_owner; unsigned int f_uid, f_gid; int f_error; unsigned long f_version; void *private_data; }; 26

28. Real-time Systems Lab, Computer Science and Engineering, ASU inode  Each object in the filesystem is represented by an inode  File type (executable, block special etc), permissions (read, write etc), Owner, Group, File Size,  File access, change and modification time, deletion time Number of links (soft/hard), extended attribute such as append only  Access Control List (ACLs)  Includes dev_t i_rdev; struct block_device *i_bdev; struct char_device *i_cdev; 27

29. Real-time Systems Lab, Computer Science and Engineering, ASU Memory Access – User and Kernel Spaces  You might receive some pointers that point to user space in your driver function  Do not dereference it directly  Access it via kernel functions ( ): copy_to_user() and copy_from_user()  They would check whether the user space pointer is valid unsigned long copy_to_user( void __user *to, const void *from, unsigned long count); unsigned long copy_from_user( void *to, const void __user *from, unsigned long count); 28

30. Real-time Systems Lab, Computer Science and Engineering, ASU The Read/Write Method 29

31. Real-time Systems Lab, Computer Science and Engineering, ASU Container_of  If strcut x contains y, can we find the pointer to x given the pointer to y  defined in : container_of(pointer, container_type, container_field);  takes a pointer to a field of type container_field, within a structure of type container_type,  returns a pointer to the containing structure. 30