1. Advanced Cubesat Imaging Payload
Joseph Mayer and Nathan Bossart
ECE 491 – Senior Design II – Department of Electrical and Computer Engineering
Background
Results
RASCAL: A two spacecraft mission to demonstrate key technologies for proximity
operations. One spacecraft will use its propulsion system in conjunction with the
imaging payload in order to facilitate orbiting and docking
Imaging Payload: To achieve the goals of the RASCAL mission, each spacecraft will identify the other and interpolate knowledge of parameters such as distance. The goal of this project is to design and implement an imaging payload for obtaining raw image data and converting it into actionable high-level data.
Completion of software verification with light and color patterns using the OpenCV libraries
Creation of a test Cubesat with six distinct patterns (with the same height and width) that are easily recognizable using computer vision and vertex-based shape detection
Initialization and interfacing with a OV7670 camera from hardware on a Zynq-7000 FPGA
Creation and verification of a basic video pipeline and initialization software (using Xilinx AXI video in/out, video timing controller, VDMA, and Zynq processing IP cores)
Creation of a boot image with a custom device tree that loads custom programmable logic (the video pipeline hardware) before booting into a PetaLinux OS ramdisk
Creation of customized HLS blocks to perform Sobel filtering and light thresholding
Work towards integration of the HLS blocks into the video pipeline to provide hardware acceleration
Work towards creation of Linux drivers for interacting with and configuring the AXI hardware blocks via the PetaLinux OS
Work towards creation of a Linux application to carry out device initialization (using custom Linux drivers) and to complete the remainder of the image processing tasks using cross-compiled OpenCV libraries
Figure 1: RASCAL mission diagram
Description
Identification Strategy:
Involves using light/color patterns on the side of a Cubesat for identification
A model Cubesat with closed polygon patterns was created to-scale for testing purposes
Each pattern has the same height and width and is recognizable using a basic vertex-based shape detection strategy
Relative distance is obtained using the scale of the object in frame
Relative angles are obtained using position of the object in frame along with the relative distance
Software Verification:
Before constructing the video pipeline on the Zynq-7000, we constructed and verified our image processing algorithms using the OpenCV libraries on low-grade hardware
Additionally, much of the code for contour simplification and vertex detection could be reused in the software domain of the final system
Processing Image Data:
A 640x480 test camera (OV7670) was bought and interfaced with the Zedboard Development Board
A video pipeline was generated using Xilinx’s Vivado tool chain for image processing in the software domain
Vivado HLS was used to generate hardware blocks to offload processing tasks better suited for hardware
Xilinx SDK was used to create Linux applications to initialize the video pipeline and carry out tasks such as contour detection and angle/distance calculations
Figure 2: Test Cubesat faces
Future Work
Integration of HLS blocks into the video pipeline*
Linux driver creation for Xilinx AXI peripherals (VDMA, video timing controllers, and custom HLS blocks)*
Cross-compilation of OpenCV program to carry out recognition tasks on Zynq core running Linux*
Creation and integration of a higher-resolution flight camera (potentially a FLIR)
In-depth calibration of algorithms and enabling of processing system in the context of the RASCAL system
Custom board creation and satellite integration
* Denotes that a significant amount of work towards this portion has already been completed
Figure 4: Software Verification Demonstration
References & Thanks
Special thanks to Dr. Kyle Mitchell, Dr. Jason Fritts, and Dr. Will Ebel.
[1] Jan Erik Solem, Programming Computer Vision with Python. Creative Commons.
[2] Milan Sonka, Vaclav Hlavac, Roger Boyle, Image Processing, Analysis, and Machine Vision. Cengage Learning; 3rd edition.
[3] October 29, 2013
[4] October 31, 2013
[5] November 11, 2013
[6] Gary Bradski, Adrian Kaehler, Learning OpenCV: Computer Vision with the OpenCV Library. O'Reilly Media, Inc.; 1st edition.
[7] Matthew Russell, Scott Fischaber, " OpenCV Based Road Sign Recognition on Zynq," th
IEEE Internation Conference On Industrial Informatics, pp , July 29, 2013
[8] February 10, 2014
[9] FMC-IMAGEON Building a Video Design from Scratch Tutorial, Avnet, Version 1.3, March 15, 2014
[10] Processor Control of Vivado HLS Designs, Fernando Martinez Vallina, XAPP745, April 7, 2014
[11] Zedboard (Zynq Evaluation and Development) Hardware User’s Guide, Avnet, Version 1.1, August 1, 2012
Figure 3: Zedboard Development Board and OV7670 Camera
Figure 5: Gantt Chart (as of 21 April 2014)
Figure 6: Team Photo – Joe Mayer, Bob Urberger, and Nathan Bossart