1. 1 1 ME2130 EMBEDDED LINUX OS (Intel® Atom™ processor) This courseware product contains scholarly and technical information and is protected by copyright laws and international treaties. No part of this publication may be reproduced by any means, be it transmitted, transcribed, photocopied, stored in a retrieval system, or translated into any language in any form, without the prior written permission of Acehub Vista Sdn. Bhd. The use of the courseware product and all other products developed and/or distributed by Acehub Vista Sdn. Bhd. are subject to the applicable License Agreement. For further information, see the Courseware Product License Agreement. http://dreamcatcher.asia/cw

2. 2 2 Linux and the Atom Author: Prof. Ian McLoughlin

3. 3 3 Course Structure and Timetable C00: Introduction and Overview C01: About Embedded Systems (1 hr) C02: Survey of Embedded Components (2 hrs) C03: Programming Models & Languages (3 hrs) C04: Introduction to the Atom Processor (4 hrs) C05: Embedded Operating Systems (3 hrs) C06: Toolchain (3 hrs) C07: Linux and the Atom (3 hrs) C08: Developing Embedded Applications (8 hrs) C09: Connectivity (3 hrs) C10: Efficient Embedded Solutions (6 hrs) Note: Timings are approximate

4. 4 4 Contents Embedded Linux Intelligent Systems The Linux Kernel – What It is, Where to Get It Exploring the Kernel Source Configuring and Compiling the Kernel Booting Different Sizes of Embedded System Linux Running and Scheduling Tasks

5. 5 5 Introduction The kernel is the core of the operating system. As we know it's not the whole OS, or maybe even the biggest bit, but it holds everything together! One of the amazing benefits of using Linux is that we can customise the entire OS, even customise the kernel: this is one of the main reasons for choosing an open source solution with building an intelligent system

6. 6 6 Embedded Linux on Our System When we want to get embedded Linux running on a system, we know we will need to get a few things working: 1.the kernel 2.a ramdisk or HDD/SSD containing boot and applications code 3.run-time libraries 4.a compiler (or cross-compiler) to build all the above 5.some patience!! We actually cover most of these in this course

7. 7 7 Obtaining a Kernel Often, the hardware vendor may supply a ready-built kernel or a ready-made Linux distribution (or maybe a third party has already ported it to the hardware). Sometimes, a hardware vendor will supply a set of kernel patches which we can apply to fresh kernel source code before building the working kernel ourselves. Very occasionally, we have to port our own kernel. It's a good feeling, in the world of Linux, to be able to say that you've built your own kernel...

8. 8 8 Obtaining a Kernel To build a kernel, first we need to get hold of the kernel source code. This can always be downloaded from: http://www.kernel.org You will find many kernels (and other things available for download). First we need to understand kernel version number. Every kernel has a version number. At the time of writing, the latest kernel is version 3.7 although most embedded Linux systems are probably based on kernel 2.6.

9. 9 9 This chart, from www.kernel.org shows a timeline of the kernel version numbers. Historically, the odd numbered versions (2.1, 2.3, 2.5) are development ones, and the even numbered ones are for general release, but that changed after 2.6. The third digit is the release number (minor revision). So 2.6.30 was release 30 of the 2.6 kernel. Any digits or letters after that are less important (they may be the initials of the developer). Obtaining a Kernel

10. 10 2.6.11 2.6.12 2.6.132.6.142.6.152.6.162.6.172.6.182.6.192.6.202.6.21 2.6.22 2.6.232.6.242.6.252.6.262.6.272.6.282.6.29 2.6.302.6.31 2.6.32 2.6.332.6.34 2.6.352.6.362.6.372.6.38 2.6.39 3.0 3.1 3.2 0 2000000 4000000 6000000 8000000 10000000 12000000 14000000 16000000 Source lines of code by kernel version Version SLOC Obtaining a Kernel

11. 11 Obtaining a Kernel But which kernel should we download? Usually, we get the newest release version unless you have some patch code, or some driver that requires a particular version. For example, if I have a patch for an ARM AT91RM9200 embedded system called 2.6.30-at91.patch.gz, I would need to download kernel version 2.6.30 to apply the patch to.

12. 12 Obtaining a Kernel We normally download the kernel in.tar.bz2 format and save in some convenient directory. tar – UNIX tape archive (a way of archiving a directory or set of files) bz2 – sometimes called 'better' zip (compression) The download is about 40 Mbytes in size, and it will expand to 100 or more Mbytes before we are finished (even though the final kernel executable might be only 800kbyes in size!) We next unpack the source code with the command: tar xjvf linux-3.2.33.tar.bz2 There is a new (better) compression scheme now, with the extension.xz

13. 13.configarch firmware modules.builtin scripts virt blockKbuild modules.order vmlinuxCOPYINGfs kernel Module.symverssecurity vmlinux.oCREDITSinclude lib net sound crypto init MAINTAINERS README System.mapDocumentation ipc Makefile REPORTING-BUGS toolsdrivers samples usrmm Documentation: useful to take a look in here! devices.txt, arch/, kernel-parameters.txt, etc... arch: device specific code net: networking code drivers: LKMs and similar init: startup code mm: memory management kernel: the kernel code Exploring the Kernel

14. 14 Modifying the Kernel Before we change anything inside the kernel, we should take a backup of the kernel configuration file. cd linux-2.6.33.2 (enter the directory) cp.config mybackup.config (or some other convenient name) It's also useful to take a brief look at the configuration file: less.config To actually configure the kernel there are a few ways, but the easiest is the one called menuconfig. Don't edit.config directly! make menuconfig Note: This assumes that your system already has the software tools for building the kernel installed (not always true for freshly installed Linux)

15. 15 Modifying the Kernel If this works, it brings up the menuconfig system for configuring the kernel: Let's choose an example to change the configuration...

16. 16 Modifying the Kernel We will try to add USB keyboard support and USB flash support step-by-step over the next few pages... this is what we will make sure is enabled: A:Device Drivers | USB EHCI HCD (USB 2.0) support(*) EHCI HCD (Most Intel....) support (*) USB human interface device (HID) support (*) B. Input (core) support Keyboard support (*) C.USB Mass storage (*) Note that configuration option names may change from kernel to kernel, but will be at least similar:

17. 17

18. 18

19. 19

20. 20

21. 21 In many cases, we have the option to enable a driver as a plug in module (an LKM),, or as a core part of the monolithic kernel itself. Usually, unless it some kind of 'always on' driver, then it's better to make it an LKM. Next, we exit and save the configuration file Now, if you looked at.config again, you would see our changes have been made. We can do this to quickly see them: diff.config mybackup.config Modifying the Kernel

22. 22 Modifying the Kernel When we are done making changes in the menus, have saved the new.config and exited the menuconfig system, we can proceed: make bzImage In a real development project we would now: check the size of the new kernel (hopefully not too big) try booting it on our system (a quick & easy test), If all works as expected, make and install the modules inside the ramdisk and copy this and the new kernel onto the boot medium (i.e. the SSD drive). So next time we reboot, the new kernel and ramdisk will be used.

23. 23 Modifying the Kernel Note for reference: This is how we make and install the modules to the ramdisk: make modules Then mount the ramdisk image on the loopback interface (huh?), and then install the modules onto the ramdisk: make modules_install INSTALL_MOD_PATH=~path_to_ramdisk

24. 24 The Kernel Source Customising the kernel and modules like this isn't difficult, but sometimes we actually have to examine and change the kernel source code itself. This is far more satisfying to hard-core kernel hackers.

25. 25 The Kernel Source For example, we can search for the part of the kernel that runs the init program... the first code that gets executed when the kernel boots up: Inside the kernel source code directory: grep -r "/bin/init" init/* If you do this, you will see some quite readable code... Although the kernel source code is HUGE and COMPLICATED, you might be surprised at how much of is readable and understandable!

26. 26 The Kernel Source Not many people go through the kernel source just because 'they want to', usually it’s in response to some type of problem such as: When porting to a new architecture Trying to get new hardware working Looking for efficiency gains Searching for security flaws and vulnerabilities

27. 27 When the Kernel Gets Run When you boot up any embedded Linux system, there are a few well-defined steps in the sequence. We've talked about these briefly before. 1.Bootloader – its role is very low level and basic system configuration, preparing everything ready for the kernel (i.e. may include loading kernel and ramdisk to memory) 2.Kernel executes 3.Kernel makes sure ramdisk is unpacked and ready to become the root filesystem, or directly mounts a hard disk (or SSD) to be the root filesystem

28. 28 The Linux kernel is an executable compressed with gzip or bzip2. It has a name vmlinux, bzimage, zimage or similar At the start of the kernel is a built in decompressor with a CRC checker. It is this that executes first... When an error is detected in the decompressed kernel, the code prints “CRC Error” and halts at that point. Let us see what it prints out when it works (next page) Note: This is for an ARM Linux system – Atom systems will differ in several ways, which we'll see as we progress through the course. Embedded Linux – Three-stage Boot

29. 29 Starting kernel... Uncompressing Linux.................................. done, booting the kernel. Linux version 2.6.4-br1 (asian@pegasus.ntu.edu.sg) (gcc version 3.4.4) #11 Mon Aug 8 16:10:29 NZST 2005 CPU: StrongARM-1110 [6901b119] revision 9 (ARMv4) Machine: X-Sat ARM Payload Memory policy: ECC disabled, Data cache writeback On node 0 totalpages: 16384 DMA zone: 16384 pages, LIFO batch:4 Normal zone: 0 pages, LIFO batch:1 HighMem zone: 0 pages, LIFO batch:1 Built 1 zonelists Kernel command line: console=ttySA0,115200n8 mem=64M root=0100 PID hash table entries: 512 (order 9: 4096 bytes) Warning: uninitialized Real Time Clock Console: colour dummy device 80x30 Memory: 64MB = 64MB total Memory: 61680KB available (796K code, 170K data, 56K init) Calibrating delay loop... 68.60 BogoMIPS Dentry cache hash table entries: 8192 (order: 3, 32768 bytes) Inode-cache hash table entries: 4096 (order: 2, 16384 bytes) Mount-cache hash table entries: 512 (order: 0, 4096 bytes) checking if image is initramfs...it isn't (no cpio magic); looks like an initrd Freeing initrd memory: 2048K CPU: Testing write buffer coherency: ok POSIX conformance testing by UNIFIX Linux NoNET1.0 for Linux 2.6 NetWinder Floating Point Emulator V0.97 (extended precision) ikconfig 0.7 with /proc/config* Real Time Clock Driver v1.12 Serial: SA11x0 driver $Revision: 1.50 $ ttySA0 at MMIO 0x80010000 (irq = 15) is a SA1100 ttySA2 at MMIO 0x80050000 (irq = 17) is a SA1100 RAMDISK driver initialized: 16 RAM disks of 2097152K size 1024 blocksize mice: PS/2 mouse device common for all mice RAMDISK: Compressed image found at block 0 VFS: Mounted root (ext2 filesystem). Freeing init memory: 56K init started: BusyBox v1.00-pre8 (2004.03.05-22:18+0000) multi-call binary What happens next? We will follow the process in more detail Starting kernel... Uncompressing Linux............. done, booting the kernel.

30. 30 kernel starts initialises memory initialises hardware initialises device drivers mount root filesystem run init program Linux version 2.6.4-br1 (asian@pegasus.ntu.edu.sg) (gcc version 3.4.4) #11 Mon Aug 8 16:10:29 NZST 2005 CPU: StrongARM-1110 [6901b119] revision 9 (ARMv4) Machine: X-Sat ARM Payload Embedded Linux – Three-stage Boot

31. 31 Memory policy: ECC disabled, Data cache writeback On node 0 totalpages: 16384 DMA zone: 16384 pages, LIFO batch:4 Normal zone: 0 pages, LIFO batch:1 HighMem zone: 0 pages, LIFO batch:1 kernel starts initialises memory initialises hardware initialises device drivers mount root filesystem run init program Embedded Linux – Three-stage Boot

32. 32 PID hash table entries: 512 (order 9: 4096 bytes) Warning: uninitialized Real Time Clock Console: colour dummy device 80x30 Memory: 64MB = 64MB total Memory: 61680KB available (796K code, 170K data, 56K init) Calibrating delay loop... 68.60 BogoMIPS Dentry cache hash table entries: 8192 (order: 3, 32768 bytes) Inode-cache hash table entries: 4096 (order: 2, 16384 bytes) Mount-cache hash table entries: 512 (order: 0, 4096 bytes) kernel starts initialises memory initialises hardware initialises device drivers mount root filesystem run init program Embedded Linux – Three-stage Boot

33. 33 Linux NoNET1.0 for Linux 2.6 NetWinder Floating Point Emulator V0.97 (extended precision) ikconfig 0.7 with /proc/config* Real Time Clock Driver v1.12 Serial: SA11x0 driver $Revision: 1.50 $ ttySA0 at MMIO 0x80010000 (irq = 15) is a SA1100 ttySA2 at MMIO 0x80050000 (irq = 17) is a SA1100 RAMDISK driver initialized: 16 RAM disks of 2097152K size 1024 blocksize mice: PS/2 mouse device common for all mice Embedded Linux – Three-stage Boot kernel starts initialises memory initialises hardware initialises device drivers mount root filesystem run init program

34. 34 RAMDISK: Compressed image found at block 0 VFS: Mounted root (ext2 filesystem). Freeing init memory: 56K init started: BusyBox v1.00-pre8 (2004.03.05-22:18+0000) multi-call binary kernel starts initialises memory initialises hardware initialises device drivers mount root filesystem run init program Embedded Linux – Three-stage Boot

35. 35 So what is the situation after it has booted? 1.The kernel (and all kernel modules and processes) launched 2.The ramdisk mounted as a filesystem: giving us the root (/) directory and everything below it. 3.The init program executes. This has only one job: to do whatever /etc/inittab tells it. In most embedded systems, init is part of Busybox – an amazing executable that incorporates almost all UNIX tool programs, utilities and low-level helper applications. Note: If the kernel can't find init, it has some backup strategies. But we will assume that everything is working properly. Embedded Linux – Three-stage Boot

36. 36 Embedded Linux – Atom Unlike the example on the previous pages, the Atom embedded system boots more like a desktop PC and ends up with a hard disk image being used as root filesystem (Note: Both can boot from USB-connected flash) Stage 1: the system turns on, RAM starts empty, the CPU begins toexecute the BIOS. Stage 2:BIOS selects boot device (the SSD system, which appears to the BIOS just like a hard disc). The MBR is read from this and executed – causing the bootloader to run. Stage 3:The bootloader executes (GRUB, syslinux, LILO etc...), selects kernel, and executes it (telling it where to get the ramdisk). Stage 4:The Linux kernel executes as normal (decompresses itself, sets of the root filesystem etc...)

37. 37 Embedded Linux – Atom If we boot from an external USB-connected hard disk or flash drive, the process is similar. In this case, the BIOS reads the MBR from the USB- connected device instead of from the internal hard disk (or SSD). In fact, the USB-connected thumbdrive (a.k.a. Pendrive) is formatted to appear exactly like a hard disk to the BIOS. Often, the boot file system will 'disappear' once the system is booted (i.e. it is used during the boot process but not thereafter). The boot system is so configurable, it can be made to do many things, but we have presented the most normal, straightforward way.

38. 38 USB/flash MBR comp. ramdisk comp. kernel SDRAM syslinux SDRAM comp. kernel USB/flash MBR comp. ramdisk comp. kernel syslinux SDRAM ramdisk mounted kernel system memory Powerdown stateready to execute kernel Kernel grabs ramdisk, Linux running Embedded Linux – Atom USB/flash device is no longer mounted to the system This is how we might boot from USB-connected storage:

39. 39 SSD MBR ramdisk comp. kernel SDRAM GRUB SDRAM comp. kernel SSD MBR ramdisk comp. kernel GRUB SDRAM kernel system memory SSD MBR ramdisk mounted comp. kernel GRUB Powerdown stateready to execute kernel Kernel grabs ramdisk, Linux running Embedded Linux – Atom For a boot from hard disk or SSD the picture is different

40. 40 Embedded Linux – Atom The BIOS runs. It reads the MBR which shows it where to find the bootloader (i.e. GRUB). The bootloader finds the kernel, prepares memory and may copy things like a ramdisk into SDRAM. Then it executes the kernel. The kernel decompresses itself and looks for a root file system. If it's a ramdisk, the kernel decompresses it if necessary, then mounts it. If it's a hard disk/SSD, the kernel simply mounts it....what happens next???

41. 41 Embedded Linux – Autorun The kernel runs init which reads /etc/inittab, performing the actions specified in that file: ::sysinit:/etc/init.d/rcS # Put a getty on the serial line (for a terminal) ::respawn:/sbin/getty -L ttyS0 115200 vt100 # And one on the main console ::askfirst:/bin/sh # Stuff to do when restarting the init process ::restart:/sbin/init # Stuff to do before rebooting ::ctrlaltdel:/sbin/reboot

42. 42 Embedded Linux – Autorun /etc/inittab told init to execute a script called /etc/init.d/rcS Here is a real-life example of one: #!/bin/sh # Mounting all devices mount -av # Mount the sys filesystem mount -t sysfs "" /sys # load the ethernet driver modprobe r8101 ifconfig eth0 192.168.0.101 up telnetd echo "Welcome to IVM system #2" Usually, we don't touch /etc/inittab but we definitely modify /etc/init.d/rcS to change the after-boot behaviour of a system.

43. 43 Embedded Linux – Running You may have noticed this comment (or similar) in the kernel boot up messages: PID hash table entries: 512 (order 9: 4096 bytes) This was setting up the PID (process ID table). The kernel does not multitask internally... Therefore, multitasking only starts once init is running. In any Linux system, you can list the running processes: ps aux It should always show init, running with PID=1.

44. 44 As soon as there is more than one program running (i.e. once init launches something which in turn launches something else), then there needs to be a way of choosing which of these programs gets to use CPU, which is a shared resource. As we have seen previously, this is the job of the scheduler. Linux Scheduling

45. 45 In Linux, whenever the scheduler runs, it chooses from the set of runnable tasks based on their priority and other metrics, and switches to the chosen task. schedule() is executed at task dispatch points in most system calls (ret_from_sys_call) and definitely by any sleep() functions encountered in running code. Linux Scheduling

46. 46 The Linux function sched_setscheduler() sets the scheduling policy and parameters for the specified PIDs. You can call this in your programs. There are three scheduling policies in standard Linux (some embedded flavours provide other alternatives, some provide none!). Often we don't touch the scheduling policy and don't touch the priorities on a desktop machine: but for efficient hard real-time embedded systems it may be necessary to explicitly set priorities and scheduling algorithms. Let's look at some... Linux Scheduling

47. 47 SCHED_OTHER: the default time-sharing preemptive scheduler that uses a PID nice level and a counter (how long has a process been waiting...) SCHED_FIFO: is first-in, first-out preemptive non-timesliced scheduler that will run a high priority process to completion, and allow newly-runnable PIDs to preempt lower priority processes SCHED_RR: adds round-robin capabilities to SCHED_FIFO. Processes don't run to completion, but are time-sliced so that other processes can run You will need root access to select these or set a higher than default process priority level. Linux Scheduling

48. 48 WARNING! Note this piece of good advice from the sched_setscheduler() man page: As a non-blocking end-less loop in a process scheduled under SCHED_FIFO or SCHED_RR will block all processes with lower priority forever, a software developer should always keep available on the console a shell scheduled under a higher static priority than the tested application. This will allow an emergency kill of tested real-time applications that do not block or terminate as expected. Linux Scheduling

49. 49 The nice command, issued at the shell prompt, can set the priority of a program to 40 levels ranging from -20 to 19. nice./myprog -5 renice can change the priority of a running program. A nice program does not use all of the CPU... so if we decrease the niceness of PID 232 like this: renice -9 232 then it will use more CPU. If we increase the niceness to +19, it will be a very nice program, sharing most of its CPU time. Linux Scheduling

50. 50 End of Module C07