Android Below Java Understanding the Underlying System Architecture, and Build Process Michael Mitchell Department of Computer Science Florida State University Tallahassee, Florida 32306, USA

Overview of Topics Terminology Android System Architecture Android Tools and Utilities Compiling the Android System Flashing firmware images Compiling the Android kernel Links to additional resources

Terminology (ROM) Technically, ROM stands for Read Only Memory, which - as it says - means that you cannot write to it, it is read only - like a DVD. Confusing, because manufacturers refer to the system partition as ROM because they do not intend for the user to write to it. – So, even though it is technically writable, it is sometimes referred to as ROM because of the way it is designed to be used. Furthermore, ROM is often used as a shorthand for "ROM image". – In other words, the files that you put in the system partition are also referred to as ROM sometimes. – Thus, you'll hear people say "flashing a ROM". Example: You can flash a ROM onto the ROM, which isn't really ROM, actually.

Terminology (Root) Like ROM, root can mean several different things, depending on the context: – 'root' is the name for administrator in Android and other Linux based systems. – 'root' also means having administrator powers. – 'root' is the lowest level one can go in a file system. In other words, it is / in Linux or C:/ in Windows. – 'rooting' is the process of acquiring root (or administrator) privileges on the device. Example: You need to root the device so that you can have root access to the root of the file system.

Terminology cont. Firmware - Programs stored in the ROM, EPROM, or flash memory that usually control various internal electronic devices (Hard Drives, Keyboards, Displays, etc). Firmware is typically 'fixed' software that is not updated in consumer devices, however it is often updated (or 'flashed') by advanced users to fix bugs or add features to the device. Kernel - The central or core software component of most operating systems. Its responsibilities include managing the system's resources (the communication between hardware and software components) and can provide the lowest-level abstraction layer for resources (especially memory, processors, and I/O devices).

Terminology cont. Radio - The cellular radio on the device which needs firmware to control it. SPL - Second Program Loader, in conjunction with the IPL comprise a device's bootloader. Aside from bootstrapping Android, the bootloader also fulfills various diagnostic functions. One of these functions is the manipulation of data in the device's internal flash ram. Depending on the SPL installed, the user might be able to flash signed NBH files, flash nand images, and more. – Note that the SPL is installed and operates independently of the Android build that runs atop it!

Terminology cont. Flashing - The process of applying a firmware image (or ROM) to a device. It generally entails a very specific order of steps. Failing to complete any one of these steps properly may result in bricking the device. Unlock - Most GSM devices are locked to only work with the sim cards of a particular carrier. Obtaining root access on the device usually does nothing to the sim-lock. The usual way to unlock the device to all SIM cards is to obtain a subsidy unlock code from a carrier. Brick - a device that no longer functions, generally caused by a failed firmware or SPL update. Since the device no longer works as intended, it is often referred to as a "brick" or "paper-weight", since that is all it is good for.

Android Tools ADB - Android Development Bridge. Command line tool used to communicate with & control the device over a USB link from a computer. – Daemon runs as root on emulator/root’d phone, otherwise very limited privileges – Can send shell commands (dd, ls, mount, cat, ps, date, uptime,uname -a, mount, etc.) – Can recursively push/pull files (logical)adp pull|push Fastboot - a protocol used to directly update the flash filesystem in Android devices from a host over USB. It allows flashing of unsigned partition images.

Android Tools flash_image - A small utility to be used on the device that allows you to re-write system partitions with image files. NANDroid - A set of tools that will enable anyone who has root on their Android device to make FULL system backups. NANDroid will backup (and restore) /system, /data, /cache, and /boot partitions. Zipalign - Reduces the amount of RAM consumed when running the application by allowing data to be mmap'd in; which causes all uncompressed data within the.apk, such as images or raw files, to be aligned on 4-byte boundaries.

Android System Architecture Android is a multiprocessing system. Each application runs on top of the Android Dalvik machine. Each Dalvik machine runs on top of a Linux process. Each process runs in its own sandbox, which means it can access only the resources it creates. To access resources outside of the application sandbox, the application needs to request permission from the Android system. – android.Manifest.permission.

System Architecture Overview

Linux Kernel Android relies on Linux version 2.6.x for core – Approximately 115 patches from mainline Provides system services, including: – security, memory management, process management, network stack, and device driver model. The kernel also acts as an abstraction layer between the hardware and the rest of the software stack.(easier access to peripheral hardware devices)

System Libraries On top of Linux is a set of libraries including bionic (the Google libc), media support for audio and video, graphics and a lightweight database, which is a useful repository for storage and sharing of application data. Provides most of the functionality available in the core libraries of the Java programming language.

Android Run-time Provides a consistent environment on which every Android application runs Each Android application runs in its own process, with its own instance of the Dalvik Virtual Machine. Relies on the Linux Kernel for underlying functionality such as: – Multi-Threading – Low-level memory management

Dalvik Virtual Machine Dalvik VM != Java VM Designed specifically for Android and is optimized in two key ways. – multiple instantiation (each application has its own private copy running in a Linux process) – register based (instead of being stack based like most Java VMs) Executes the Dalvik Executable (.dex) – optimized for minimal memory footprint.

Processes & Lifecycles Components share a common Linux process: – by default, one process per.apk file..apk files are isolated and communicate with each other via Intents or AIDL. When the first of an application's components needs to be run, Android starts a Linux process for it with a single thread of execution (Main Thread). – Android tries to maintain a process for as long as possible, – eventually will need to remove old processes when memory runs low. To determine candidates to be killed, Android places each process into an "importance hierarchy" based on the components running in it and the state of those components.

Android Device Partitions Android devices have their internal flash memory divided into partitions. Common Partitions – splash1: This partition contains the very first image you see when you turn on the device. – boot: This partition stores the kernel. It also controls other aspects of the operating system on the device, such as what file formats the device is able to support. – recovery: This partition holds the device's recovery software, which gives the user access to special functions, such as upgrading the operating system, or making a backup of the device's software. – system: This is the main partition where the device's operating system is stored. Generally speaking, this partition is write-protected, and only able to be modified under special circumstances. – cache: This partition stores temporary files. Generally, files on this partition are not critical to the operation of the device. – data: This partition is where all of your data is stored. If you download an app from the Android Market, it goes here. If you change a setting, that change is stored here. If you receive a SMS, it is stored here.

Android file systems adb shell # mount rootfs on / type rootfs (ro,relatime) tmpfs on /dev type tmpfs (rw,relatime,mode=755) devpts on /dev/pts type devpts (rw,relatime,mode=600) proc on /proc type proc (rw,relatime) sysfs on /sys type sysfs (rw,relatime) none on /acct type cgroup (rw,relatime,cpuacct) tmpfs on /mnt/asec type tmpfs (rw,relatime,mode=755,gid=1000) tmpfs on /mnt/obb type tmpfs (rw,relatime,mode=755,gid=1000) none on /dev/cpuctl type cgroup (rw,relatime,cpu) /dev/block/mtdblock3 on /system type yaffs2 (ro,relatime) /dev/block/mmcblk0p1 on /data type ext3 (rw,nosuid,noatime,nodiratime,barrier=1,nodelalloc,data=ordered) /dev/block/mtdblock6 on /data/data type yaffs2 (rw,nosuid,nodev,relatime) /dev/block/mmcblk0p2 on /cache type ext3 (rw,nosuid,nodev,noatime,nodiratime,barrier=1,nodelalloc,data=ordered)

Android MTD blocks adb shell # cat /proc/mtd dev: size erasesize name mtd0: 000a "misc" mtd1: "recovery" mtd2: "boot" mtd3: 0f "system" mtd4: 000a "local" mtd5: "cache" mtd6: "datadata"

Android boot.img | boot header | 1 page | kernel | n pages | ramdisk | m pages | second stage | o pages n = (kernel_size + page_size - 1) / page_size m = (ramdisk_size + page_size - 1) / page_size o = (second_size + page_size - 1) / page_size

Android boot.img A ramdisk is a small filesystem containing the core files needed to initialize the system. It includes the critical init process, as well as init.rc, which is where you can set many system- wide properties. Files on a typical ramdisk:./init.trout.rc./default.prop./proc./dev./init.rc./init./init.goldfish.rc./sbin./sbin/adbd./system./data

Compile Android System Android system is complete compilation contains the following: – Host tools; Target system, boot, and data image files; Target Linux kernel (compiled separately); Installable zip file to flash in Recovery mode Basic Process: – Install Dependencies – Acquire the Source – Copy proprietary files – Configure & Compile

Install Dependencies For 32/64-bit systems: sudo add-apt-repository "deb maverick partner“ sudo apt-get update sudo apt-get install git-core gnupg flex bison gperf libsdl1.2- dev libesd0-dev libwxgtk2.6-dev squashfs-tools build- essential zip curl libncurses5-dev zlib1g-dev sun-java6-jdk pngcrush schedtool For 64-bit systems only : sudo apt-get install g++-multilib lib32z1-dev lib32ncurses5-dev lib32readline5-dev gcc-4.3-multilib g multilib Get the repo tool curl > ~/bin/repo chmod a+x ~/bin/repo

Acquire the Source Tree Make a directory for the source tree mkdir ~/android cd ~/android Use the repo tool to initialize and sync the remote/local repositories repo init -u git://github.com/CyanogenMod/android.git -b gingerbread repo sync -j16 ~123,694 items, requires about 3.0 GB storage

Copy Proprietary Files While android is almost entirely open-source certain components remain proprietary & closed – Mostly hardware related – Must be copied from current system builds Navigate to your specific device cd ~/android/device/[vendor]/[device]/./extract-files.sh Get Rom-manager and Google App files cd ~/android/vendor/cyanogen./get-rommanager./get-google-files NOTE: If some hardware isn't working, like camcorder or FM radio, you will need to find the updated prop blobs.

Configure & Compile Setup the build environment. build/envsetup.sh Use the ‘brunch’ utility to select build target brunch [ target ] Takes ~1 - 3 hours to complete After compiled: ~172,562 items, requires about 6.6 GB storage

Compilation Results Compiled host utilities: ~/android/out/Host/Linux-x86/bin Compiled target files: ~/android/out/target/Product/[target] – ramdisk. img [root file system image] – root [root file system directory] – System [master file system directory] – System. img [main file system image] – UserData - qemu. img [ QEMU data image] – UserData. img [image data]

Flash the system image Make a NANDroid backup Copy your.zip file from: ~/android/out/target/product/[target]/update.cm- XXXXX-signed.zip to the root of the SD card. Download Google Apps and place at root of the SD card. (optional, but recommended) Reboot into device Recovery mode Seriously, make a NANDroid backup Flash one/both of these.zip files from recovery. NOTE: Be patient, Sometimes this may take a VERY long time!

Loading source in eclipse Make sure you have source and your build completed without errors! Copy the.classpath file to the root folder of your source cp ~/bin/eclipse/.classpath ~/android/ Open Eclipse – Click the File menu » new » java project – Give a project name like Cyanogen – Uncheck use default location – Point the location to ~/android – Keep all other defaults & Click finish

Download the kernel source Make a directory to store the kernel tree git clone git://github.com/CyanogenMod/cm- kernel.git 38,463 files, requires about MB View other available branches: git branch Checkout a branch: git checkout –b ‘branch_name’

Configure the Build Pull the.config from the newest boot.img scripts/extract-ikconfig boot.img >.config Alternatively, retrieve a working kernel config from the device, and unzip it: adb pull /proc/config.gz /home/michael/android/kernel/cm-kernel/config.gz cat config.gz | gunzip >.config Just enter to accept defaults, or customize as needed: make ARCH=arm CROSS_COMPILE=$CCOMPILER oldconfig make ARCH=arm CROSS_COMPILE=$CCOMPILER menuconfig

Compile the kernel Make the build make ARCH=arm CROSS_COMPILE=$CCOMPILER –j5 This step may take a while (but not nearly as long as the system compilation Finally, you should have a kernel stored in ~/android/kernel/cm-kernel/arch/arm/boot/zImage

Compiling kernel modules After installing a custom kernel, the wifi module may be unstable or unusable. The solution is to recompile the module, linking it to the new kernel build. cd ~/android/system/system/wlan/ti/sta_dk_4_0_4_32 KERNEL_DIR=~/android/kernel CROSS_COMPILE=$CCOMPILER ARCH=arm make –j5 A file named wlan.ko will be produced in the current directory. You must install the new module to the device. adb shell mount -o remount,rw /system adb shell cp /system/lib/modules/wlan.ko /system/lib/modules/wlan.ko.backup adb push wlan.ko /system/lib/modules/wlan.ko Reboot the device. If all goes well, you should be using the newly compiled 'wlan.ko'.

Additional Info & References uilding_Kernel_from_source uilding_Kernel_from_source dls.com/wiki/index.php?title=HOWTO:_Unpack,_ Edit,_and_Re-Pack_Boot_Images dls.com/wiki/index.php?title=HOWTO:_Unpack,_ Edit,_and_Re-Pack_Boot_Images ml ml