Media Player for the i.MX31 Advanced Embedded Systems Architecture Class Project May 14, 2011 Rafael Castro Ryan Ugland Carlos Cabral

Problem Definition/Goal The goal of our project was to build a media player application using the i.MX31 board that could play mp3 files. May 14, Univ. of Texas at Austin

Current State of the Art Many different media players exist today in different forms (phones, tablets, dedicated mp3 players, cars stereos, etc.) We did not expand on the current state of the art, rather we were exploring the process and challenges involved in creating a media player app and running it on existing hardware. May 14, 2011 Univ. of Texas at Austin 3

Freescale i.MX31 PDK 3-Stack Platform (CPU, Personality and Debug boards) i.MX31 ARM 11 Processor 128 MB DDR SDRAM 256 MB NAND Flash Audio 2.7” TFT Display Touch Controller May 14, Univ. of Texas at Austin

Project Definition and Tasks Setup Build Environment (Everyone) ◦Installing and building LTIB for generating base Linux image for board ◦Setting up rootfs through NFS/TFTP for accessing file system on host machine Setup multi-media packages necessary for the player application (Everyone, but Rafael succeeded so we used his system) ◦Gstreamer and codecs ◦DirectFB graphics library ◦GUI Packages (Glade, Qtopia, GTK+, etc.) Develop GUI Application (Ryan) ◦DirectFB GUI with buttons May 14, 2011Univ. of Texas at Austin 5

System Block Diagram May 14, 2011Univ. of Texas at Austin 6 i.MX31 3-Stack Board Host Linux Machine Serial -Console Messages Ethernet -File System (rootfs) -Communication (NFS/TFTP)

Setup Build Environment Linux Target Image Builder (LTIB) ◦Generate Linux image for embedded system NFS (Network File System) and Rootfs ◦Embedded system accesses the rootfs on the Linux host machine via NFS TFTP (Trivial File Transfer Protocol) ◦Used to load kernel image May 14, 2011 Univ. of Texas at Austin 7

LTIB (Linux Target Image Builder) Open source project that provides a utility to develop and deploy BSP’s (Board Support Package) for various target platforms Final result is an image file that is transferred to to the embedded system (i.MX31) May 14, 2011 Univ. of Texas at Austin 8

LTIB Image Configuration May 14, 2011 Univ. of Texas at Austin 9

Many Issues During LTIB Compile! The LTIB image generation is NOT a simple “click and go” process. Various compile problems: ◦Missing packages (.deb vs..rpm packages) ◦Broken scripts ◦Incompatible versions ◦Cache of intermediate state (difficult to debug) ◦Permissions problems May 14, 2011Univ. of Texas at Austin 10

NFS (Network File System) Allows a user on a client computer to access files over a network Was used to allow the i.MX31 embedded system to access the root file system (rootfs) on the host Linux machine Preserves flash since limits writes to the flash. Changes to the rootfs are done on the Linux host May 14, 2011Univ. of Texas at Austin 11

TFTP (Trivial File Transfer Protocol) Simple file transfer protocol generally used for automated transfer of configuration or boot files between machines in a local environment. Was used to transfer the kernel image from the host Linux machine to the i.MX31 embedded system May 14, 2011Univ. of Texas at Austin 12

Multi-media Packages All open source packages for Linux Many ports for embedded platforms available Enabled different options for app. development Difficult to setup. Easy to use! ◦GStreamer ◦Tslib ◦GTK+ ◦Glade3 ◦Qtopia May 14, 2011 Univ. of Texas at Austin 13

GStreamer Framework for creating streaming media applications (audio, video, etc) Provides: ◦ an API for multi-media applications ◦a Plugin architecture ◦a pipeline architecture (to connect plugins) ◦a mechanism for media type handling/negotiation Port of GStreamer to IMX31 available from Freescale May 14, 2011 Univ. of Texas at Austin 14

GStreamer Example May 14, 2011 Univ. of Texas at Austin 15 Example: gst-launch filesrc location=Kaleidoscope.avi ! mfw_avidemuxer ! mfw_mpeg4decoder ! mfw_v4lsink EDIOL_2010MAR16_MCP_AN_01.pdf?SOURCES=DOWNLOAD

TSLib Tslib is an abstraction layer for touchscreen panel events, as well as a filter stack for the manipulation of those events Includes routines for finding and applying calibration values Calibration values stored in file (pointed by TSLIB_CALIBFILE env. variable) Source code found at: May 14, 2011 Univ. of Texas at Austin 16

TSLib Calibration is very important!! Couldn’t get GTK+ to read calibration values Works with Qtopia May 14, 2011 Univ. of Texas at Austin 17

GTK+ Framework for creating graphical user interfaces with cross platform compatibility. ◦Glib: Main C library for graphical environment. Provides data structures and API to create graphics and handle events. ◦Pango: A library for layout and rendering of text ◦Cairo (2D graphics), ATK (accessibility) etc… Cross-platform means you can develop in your PC and then deploy to embedded target May 14, 2011 Univ. of Texas at Austin 18

Glade3 Glade3 is a RAD (Rapid Application Development) tool for designing GTK+ applications. Basically: ◦Draw user interface. ◦Define callbacks for specific events ◦Creates XML file (“glade file”) ◦Compile program ◦Run, using Libglade library May 14, 2011 Univ. of Texas at Austin 19

Glade3, GTK+ May 14, 2011 Univ. of Texas at Austin 20 Glade Interface -> <- Example of GTK+ Text editor Nice tutorial:

Qtopia (aka Qt …) Full environment for Mobile/PDK Implements virtual machine with: ◦Own virtual file system ◦Own windowing system ◦Java integration… Built on top of GTK+, TSLib, GStreamer… Has changed names, groups etc. ◦Currently a version supported by Nokia (Qt) May 14, 2011 Univ. of Texas at Austin 21

Qtopia Port of Qtopia available for IMX31 Includes a Media Player! Made it work in our system. Decided to develop our own project outside Qtopia Source: May 14, 2011 Univ. of Texas at Austin 22

DirectFB Overview DirectFB is a an open source software library that provides access to the linux framebuffer. Built in support for touchscreens and keyboards. Used in Palm / HP WebOS. GTK+ apps can be run through DirectFB rather than X-Windows on embedded devices. Support for OpenGL May 14, 2011 Univ. of Texas at Austin 23

DirectFB Media Player Displays MP3 file list to screen Keypad up and down keys used to select MP3 Keypad enter key used to play selected MP3 Images can also be displayed to screen May 14, 2011 Univ. of Texas at Austin 24

DirectFB Media Player Development Coded in C Used DirectFB for displaying MP3 names and graphics to screen GUI developed on host machine using X- Windows emulation of Linux framebuffer, and then recompiled on target machine. ◦Only compile flags and media paths needed to be changed between host and target machine compiles Gstreamer library and codecs used for playing the MP3 files May 14, 2011 Univ. of Texas at Austin 25

Issues with DirectFB Gui DirectFB keyboard support did not work with I.MX31 keypad. Touchcreen calibration did not work with DirectFB. Decided not to use touchscreen controls. Disabled DirectFB keyboard support, and accessed the keypad driver directly ◦/dev/input/event0 Dumped all of the keypad event data to the terminal and determined how to uniquely identify each key. May 14, 2011 Univ. of Texas at Austin 26

Results and Further Work We were able to successfully create a media player application to play mp3’s. The i.MX31 PDK has many features such as support for various media codecs and a touch screen LCD. ◦Additional goals can be to allow the app to play various media (ex. mpeg files) ◦Further development of the GUI  Advanced menus and eye candy ◦Utilize the touch screen May 14, 2011Univ. of Texas at Austin 27

References Freescale i.MX31 PDK Product Website for Documentation, Source Code and Codecs ◦IMX_PDK15_LINUX_BSP (Board Support Package Download) ◦PDK 1.5 i.MX35 Linux User’s Guide ◦PDK 1.5 i.MX31 Linux Reference Manual ◦Linux Image Target Builder (LTIB) Documentation “Building a Simple 3D Media Player for the i.MX31 in Linux”, Application Note by Multimedia Applications Division, Freescale Semiconductor, Inc., Austin, Texas. Freescale Online Forums ◦ Timesys LinuxLink Development Center for Freescale i.MX31 ◦ Various internet sources for general Linux reference on topics such as package management, NFS, and TFTP server installation and setup. May 14, 2011Univ. of Texas at Austin 28