Nathan Bossart, Joe Mayer, Bob Urberger RASCAL ACIP

Team Introduction   Facilitator: Bob Urberger  Computer Engineering majors  Space Systems Research Lab

RASCAL Mission  “Rascal is a two-spacecraft mission to demonstrate key technologies for proximity operations…”  “After the on-orbit checkout, one 3U spacecraft is released and passively drifts away.... After a suitable distance, the released spacecraft will activate its propulsion system and return to within a few meters of the base. The second spacecraft will be released and the process repeated...” RASCAL ACIP

Imaging Payload  Awareness of Cubesat Environment Computer Vision  Low-Level Processing Object Detection Distance Determination  High-Level Data Navigation Thruster Control RASCAL ACIP

Functional Breakdown Unique Face Identifier Capture Images Transfer Data Process Images Output/Store Control Data RASCAL ACIP Known Pattern Raw Data Structured Data High-Level Data

Modules RASCAL ACIP LEDs Camera Computational Hardware Unique Face Identification Capture Images Transfer Data Process Images Output/Store Control Data

LEDs  Required to perform classification Not enough detail visible for other features  Three approaches Unique pattern of LEDs for each face Unique combination of colors for each face Both unique patterns and colors  Unique pattern works regardless of camera spectrum Fails when face partially visible  Color combinations only work with visible spectrum cameras Can classify cube corners as well as faces with well chosen color patterns RASCAL ACIP

Camera  Potential Camera Choices:  FLIR Tau x480, 14-bit  Visible spectrum image sensor 2-5MP - 16 to 24 bit color  Parallel data output from cameras  Component Requirements:  FLIR PCB integration Control signaling simple Low resolution, monochromatic 16.1 MB/s input data 30Hz  Visible spectrum Requires lens fixture Complex control signaling High resolution, wide color range 2MP with 24 bit color: ○ 57.6 MB/s input data 30Hz RASCAL ACIP

Processing Hardware  Processing blocks in hardware  Caching and system control managed in software  Timing and Gate Consumption  Alternatives: Pure software implementation Pure hardware implementation RASCAL ACIP

Imaging Functions Image Processing Pre- Processing Distance Detection Object Detection Object Classification RASCAL ACIP Image Data Structure Distance Data Image Edges Objects In Frame

Preprocessing  Noise suppression  Color Conversion  Object enhancement  Image segmentation  Conversion and downsampling RASCAL ACIP

Distance Detection  Identify depth from a single image  Monocular Cues Relative size Comparison of imaged objects to known shape scale at particular depth  Structured geometry identified should be easy to identify scale regular structure Square or equilateral triangle RASCAL ACIP

Distance Detection in RASCAL  Square LED pattern on spacecraft face Critical point identification Homography estimation Projective transform Point correspondence for scale  Hardware Domain Parallel matrix multiplication RASCAL ACIP

Object Detection  Identifying Objects in an Image  Region or Contour Based  Edge Detection Relies heavily on Pre- processing RASCAL ACIP (Columbia University)

Object Detection in RASCAL  Hardware Domain Canny/Deriche Sobel Operator  Constraints Cubesat size Environmental Resolution RASCAL ACIP (Columbia University)

Objection Classification  Post-object detection / image segmentation  Support vector machine (metric space classification)  Assign a class based upon pre- programmed control data RASCAL ACIP

Object Classification in RASCAL RASCAL ACIP  Completed with bare- metal software  ARM Assembly / C  Minimum distance principle (efficient)  Multi-tiered and/or multi- dimensional space from attributes given  Determine a number of attributes with significant differences between faces  Testing: expect a very high level (>95%) of correct classifications

Constraints of Object Classification  Must work with a variety of backgrounds (Earth, Moon, Sun, Space, etc.)  Ideally real time (bounded) and low latency  Updated at >=10 Hz  Must function with different sizes (patterns can vary from a few pels to larger than the frame)  Definitive discrimination functions with high reliability  Alternative algorithm: neural nets, fuzzy logic

Output to Control  System will output calculated information about placement, attitude, distance, etc.  In the future, a separate team will construct a system to interpret data and convert to control signals/data  Since this is out of the scope of our project, the output format/setup is ultimately our choice RASCAL ACIP

Functional Testing  Output Unique Pattern  Capture Images Stream Images Verify Control Signals  Transfer Data Oscilloscope Frame Buffer RASCAL ACIP  Process Image Software Verification Hardware Verification  Output/Store Data Buffer

System Testing  Camera integration  Hardware timing constraints  Block connectivity verification Blocks signal each other as intended  Full pipeline simulation Blocks interact as expected  Physical synthesis testing Data produced from each frame RASCAL ACIP

Timeline RASCAL ACIP

Estimated Costs  Designed for very low budget and small amount of needed materials  Largely out of SSRL funding FunctionPartLow EstimateHigh EstimateNotes Obtain vision dataCamera$4,000$10,000SSRL funding Camera Specification$0$10,000SSRL funding Display patternsLEDs$1$10SSRL funding Algorithm ResourcesBooks$0 library & creative commons Development ToolsZedboard$0 donation Xilinx Vivado$0 donation Oscilloscope$0 provided Desktop PC$0 provided TOTAL COST:$4,001$20,010 RASCAL ACIP

Future Work for Integration  Thruster control system  Placement into spacecraft  Radiation, vibration, space-readiness  We will provide thorough documentation for future groups RASCAL ACIP

Bibliography   Jan Erik Solem, Programming Computer Vision with Python. Creative Commons.  Dr. Ebel, Conversation  Dr. Fritts, Conversation  Dr. Mitchell, Conversation  Milan Sonka, Vaclav Hlavac, Roger Boyle, Image Processing, Analysis, and Machine Vision. Cengage Learning; 3rd edition.  IS/node14.html IS/node14.html RASCAL ACIP

Questions? RASCAL ACIP