1. Image Processing for Remote Sensing Matthew E. Nelson Joseph Coleman

2. ImaP-RS One method for detecting plant health is to look in the Near Infra-red band (700 ~ 1000 nm). A camera system was developed by the SSCL for this type of remote sensing application. This project continued to improve the efficiency and functionality of the system by moving to a FPGA based system

3. ImaP-RS Goals  Implement a working Linux kernel on the ML507 with USB Support  Cross Compile existing tools to the PowerPC architecture the ML507 uses gPhoto2 libgphoto2  Implement a co-processor design to handle the image processing

4. ImaP-RS Linux on the ML507  Xilinx provides a reference base system for installing a Linux kernel on the ML507 Limited functionality, but good starting point USB is not enabled in the base system  ML507 Cypress CY7C67300 chipset NOT supported by Linux   Plan B Digilent SMSC EVB-USB3300-XLX Daughter board It is supported by Linux

5. ImaP-RS Demo – Booting Linux on the ML507

6. ImaP-RS USB on the ML507

7. ImaP-RS USB on the ML507

8. ImaP-RS Cross Compiling the Tools  Two software packages are critical for the communication between the Canon 5D and Linux libgphoto ○ Provides camera interface libraries gphoto ○ Provides command line tools libexif optional libpopt libusb Required for USB cameras libgphoto2 gphoto2

9. ImaP-RS Cross Compiling the software  Cross Compiling libgphoto2 and gPhoto2 proved to be more difficult than originally planned ELDK would not compile libgphoto2 Would not compile against a library it had, others trying to compile reported similar problems, either ELDK or libgphoto2 problem  In addition, one other software package would be helpful UFRaw ○ Can convert from Canon’s RAW format to PNG or TIFF ○ Can also do some limited image manipulation ○ This was also difficult to cross compile due to a number of dependencies required

10. ImaP-RS Image Processing  When gPhoto is in place, we can obtain images directly from the Canon 5D We can also send commands to the camera for camera settings and to trigger a photograph  Images will be stored on a mounted CF Card

11. ImaP-RS Image Processing  Considerations  File Formats  Selection Process  Results  Connectivity  Simulink

12. ImaP-RS Image Processing Considerations  Need to get pictures off of flash drive in camera  Cannot remove the flash drive because of application environment (tethered balloon at ~50-500 ft.  Need high reliability and scripting capabilities to facilitate autonomous action  Onboard image processing for immediate data availability  Low power consumption for extended sessions

13. ImaP-RS Canon 5D Supported Image File Formats:  RAW  TIFF  JPEG UFRaw Supported Image File Formats:  PNG  TIFF  JPEG Matlab Imread( ) Supported Image File Formats:  BMP  JPEG  PNG  TIFF  GIF  Etc.

14. ImaP-RS Format Selection Criteria:  Non-compressed for intuitive pixel- based processing  Compatible with all platforms  Open standard for GPS information insertion later

15. ImaP-RS Results:  TIFF selected for image format  FPGA to be used for any filter operations  Power PC running Linux to be used for any Matlab serial image functions (loading image, flattening, etc.)  USB interface to Canon EOS 5D  Simulink to be used with Xilinx System Generator (sysgen) for DSP

16. ImaP-RS Dataflow Diagram – Hardware  Standalone ML507 does not support USB host mode  Camera does not support USB mass storage device mode Digilent USB Daughterboard CF Camera Memory (kept empty) Canon EOS 5D ML507 Power PC Linux Xilinx custom kernel version 2.6.30 Device USB Host 8-bit Data Bus

17. ImaP-RS Dataflow Diagram – Software Side ML507 Power PC Linux Xilinx custom kernel version 2.6.30 GPhoto HD CF ML507 Memory RAMRAM Notify Trigger Writes C-Program Imread( ) Rgb2gray( ) Execution Control Imwrite( ) FPGA

18. ImaP-RS Rgb2gray( ) 3D Truecolor Image Data Array 2D Grayscale Image Data Array Grayscale Image Creation Steps: Imread( ) parses file and creates a 3D matrix Rgb2gray( ) flattens the image to 2D Grayscale Intensity = 0.2989 * R + 0.5870 * G + 0.1140 * B

19. ImaP-RS Simulink with Xilinx Sysgen for DSP benefits:  Built in simulink blocks for FIR filters, FFTs, error correction, arithmetic, memory access (RAM, etc.) & digital logic  Automatic VHDL code generation for specific Xilinx IP cores  Hardware co-simulation

20. ImaP-RS

21. Xilinx FPGA boundaries in Sysgen

22. ImaP-RS Problems  Non-linux supported USB chipset on the ML507  Issue with cross compiling libgphoto  Xilinx Sysgen for DSP availability  Lack of documentation  Too much documentation  Closed protocol on Canon Cameras

23. ImaP-RS Future work  We plan to continue this work Finish working on getting the camera talking to the FPGA Integrate image processing into the ML507 board Fine tune settings

24. ImaP-RS Thank you Any Questions?

25. ImaP-RS References Xilinx Application Note 1140 http://www.xilinx.com/support/documentation/application_notes/xapp1140.pdf XPS USB Host Controller Driver http://xilinx.wikidot.com/usb-host-controller-driver Configuring, Building and Loading PowerPC Linux http://xilinx.wikidot.com/powerpc-linux gPhoto2 http://gphoto.sourceforge.net/ Matlab documentation http://www.mathworks.com/ Xilinx System Generator Documentation http://www.xilinx.com/tools/sysgen.htm Wikipedia Graphic format comparison http://en.wikipedia.org/wiki/Comparison_of_graphics_file_formats