1. Nathan Bossart, Joe Mayer, Bob Urberger RASCAL ACIP

2. Team Introduction  http://acip.us  Facilitator: Bob Urberger  Computer Engineering majors  Space Systems Research Lab

3. RASCAL Mission  “Rascal is a two-spacecraft mission to demonstrate key technologies for proximity operations…”  “After the on-orbit checkout, one 3U spacecraft is released and passively drifts away.... After a suitable distance, the released spacecraft will activate its propulsion system and return to within a few meters of the base. The second spacecraft will be released and the process repeated...” RASCAL ACIP

4. Imaging Payload  Awareness of Cubesat Environment Computer Vision  Low-Level Processing Object Detection Distance Determination  High-Level Data Navigation Thruster Control RASCAL ACIP

5. Functional Breakdown Unique Face Identifier Capture Images Transfer Data Process Images Output/Store Control Data RASCAL ACIP Known Pattern Raw Data Structured Data High-Level Data

6. Modules RASCAL ACIP LEDs Camera Computational Hardware Unique Face Identification Capture Images Transfer Data Process Images Output/Store Control Data

7. LEDs  Required to perform classification Not enough detail visible for other features  Three approaches Unique pattern of LEDs for each face Unique combination of colors for each face Both unique patterns and colors  Unique pattern works regardless of camera spectrum Fails when face partially visible  Color combinations only work with visible spectrum cameras Can classify cube corners as well as faces with well chosen color patterns RASCAL ACIP

8. Camera  Potential Camera Choices:  FLIR Tau 640 640x480, 14-bit  Visible spectrum image sensor 2-5MP - 16 to 24 bit color  Parallel data output from cameras  Component Requirements:  FLIR PCB integration Control signaling simple Low resolution, monochromatic 16.1 MB/s input data rate @ 30Hz  Visible spectrum Requires lens fixture Complex control signaling High resolution, wide color range 2MP with 24 bit color: ○ 57.6 MB/s input data rate @ 30Hz RASCAL ACIP

9. Processing Hardware  Processing blocks in hardware  Caching and system control managed in software  Timing and Gate Consumption  Alternatives: Pure software implementation Pure hardware implementation RASCAL ACIP

10. Imaging Functions Image Processing Pre- Processing Distance Detection Object Detection Object Classification RASCAL ACIP Image Data Structure Distance Data Image Edges Objects In Frame

11. Preprocessing  Noise suppression  Color Conversion  Object enhancement  Image segmentation  Conversion and downsampling RASCAL ACIP

12. Distance Detection  Identify depth from a single image  Monocular Cues Relative size Comparison of imaged objects to known shape scale at particular depth  Structured geometry identified should be easy to identify scale regular structure Square or equilateral triangle RASCAL ACIP

13. Distance Detection in RASCAL  Square LED pattern on spacecraft face Critical point identification Homography estimation Projective transform Point correspondence for scale  Hardware Domain Parallel matrix multiplication RASCAL ACIP

14. Object Detection  Identifying Objects in an Image  Region or Contour Based  Edge Detection Relies heavily on Pre- processing RASCAL ACIP (Columbia University)

15. Object Detection in RASCAL  Hardware Domain Canny/Deriche Sobel Operator  Constraints Cubesat size Environmental Resolution RASCAL ACIP (Columbia University)

16. Objection Classification  Post-object detection / image segmentation  Support vector machine (metric space classification)  Assign a class based upon pre- programmed control data RASCAL ACIP

17. Object Classification in RASCAL RASCAL ACIP  Completed with bare- metal software  ARM Assembly / C  Minimum distance principle (efficient)  Multi-tiered and/or multi- dimensional space from attributes given  Determine a number of attributes with significant differences between faces  Testing: expect a very high level (>95%) of correct classifications

18. Constraints of Object Classification  Must work with a variety of backgrounds (Earth, Moon, Sun, Space, etc.)  Ideally real time (bounded) and low latency  Updated at >=10 Hz  Must function with different sizes (patterns can vary from a few pels to larger than the frame)  Definitive discrimination functions with high reliability  Alternative algorithm: neural nets, fuzzy logic

19. Output to Control  System will output calculated information about placement, attitude, distance, etc.  In the future, a separate team will construct a system to interpret data and convert to control signals/data  Since this is out of the scope of our project, the output format/setup is ultimately our choice RASCAL ACIP

20. Functional Testing  Output Unique Pattern  Capture Images Stream Images Verify Control Signals  Transfer Data Oscilloscope Frame Buffer RASCAL ACIP  Process Image Software Verification Hardware Verification  Output/Store Data Buffer

21. System Testing  Camera integration  Hardware timing constraints  Block connectivity verification Blocks signal each other as intended  Full pipeline simulation Blocks interact as expected  Physical synthesis testing Data produced from each frame RASCAL ACIP

22. Timeline RASCAL ACIP

23. Estimated Costs  Designed for very low budget and small amount of needed materials  Largely out of SSRL funding FunctionPartLow EstimateHigh EstimateNotes Obtain vision dataCamera$4,000$10,000SSRL funding Camera Specification$0$10,000SSRL funding Display patternsLEDs$1$10SSRL funding Algorithm ResourcesBooks$0 library & creative commons Development ToolsZedboard$0 donation Xilinx Vivado$0 donation Oscilloscope$0 provided Desktop PC$0 provided TOTAL COST:$4,001$20,010 RASCAL ACIP

24. Future Work for Integration  Thruster control system  Placement into spacecraft  Radiation, vibration, space-readiness  We will provide thorough documentation for future groups RASCAL ACIP

25. Bibliography  http://cubesat.slu.edu/AstroLab/SLU-03__Rascal.html http://cubesat.slu.edu/AstroLab/SLU-03__Rascal.html  Jan Erik Solem, Programming Computer Vision with Python. Creative Commons.  Dr. Ebel, Conversation  Dr. Fritts, Conversation  Dr. Mitchell, Conversation  Milan Sonka, Vaclav Hlavac, Roger Boyle, Image Processing, Analysis, and Machine Vision. Cengage Learning; 3rd edition.  http://www.cs.columbia.edu/~jebara/htmlpapers/UTHES IS/node14.html http://www.cs.columbia.edu/~jebara/htmlpapers/UTHES IS/node14.html RASCAL ACIP

26. Questions? RASCAL ACIP