RIT Senior Design Project D3 Engineering Camera Platform Friday November 6, :00am to 11:00am

Team Members Gregory Hintz (EE) – Project Manager Samuel Skalicky (CE) – Lead Engineer, FPGA Board Jeremy Greene (EE) – Connector Board Jared Burdick (EE) – Power Michelle Bard (ME) – Environmental Tony Perrone (ME) – Physical Design

Advisors Scott Reardon (D3 Engineering) Kevin Kearney (D3 Engineering) Dr. Robert Kremens (RIT-Imaging Science) Philip Bryan (RIT – Industry Guide)

Project Status Risks BOM Analysis Feasibility Designs Test Plans

Schedule for the Design Review Overview – Gregory Hintz Electrical Discussion Processor Board and FPGA – Samuel Skalicky Connector Board, INS System – Jeremy Greene Mechanical Discussion System Design – Tony Perrone Environmental Concerns – Michelle Bard

What is the Customer Looking for? Integrate supplied components Ruggedized Unit Flight-capable package Can record and transmit Capable of processing

Black Box System Model

1.Integrate supplied components A.10MP Visual Band Camera B.1.3MP IR Camera C.Spatial Sensors i.NovAtel OEM Board OEMV3 ii.NovAtel OEM Board OEMV2 D.Camera Processing Board 2.Capture data from two cameras 3.Capture 1fps 4.Capture 30fps 5.Capture INS 30/sec (simultaneously) Camera Components Customer Needs Met 6.External INS units 7.Data processing (overlay) 8.Real time viewing 9.Store full-res. Data during flight 10. Support NovAtel GNSS board

Electronic System

1.Integrate supplied components A.10MP Visual Band Camera B.1.3MP IR Camera C.Spatial Sensors i.NovAtel OEM Board OEMV3 ii.NovAtel OEM Board OEMV2 D.Camera Processing Board 2.Capture data from two cameras 3.Capture 1fps 4.Capture 30fps 5.Capture INS 30/sec (simultaneously) Camera Components Customer Needs Met 6.External INS units 7.Data processing (overlay) 8.Real time viewing 9.Store full-res. Data during flight 10. Support NovAtel GNSS board

Final Output 10mp Camera

Electronic System

1.Integrate supplied components A.10MP Visual Band Camera B.1.3MP IR Camera C.Spatial Sensors i.NovAtel OEM Board OEMV3 ii.NovAtel OEM Board OEMV2 D.Camera Processing Board 2.Capture data from two cameras 3.Capture 1fps 4.Capture 30fps 5.Capture INS 30/sec (simultaneously) Camera Components Customer Needs Met 6.External INS units 7.Data processing (overlay) 8.Real time viewing 9.Store full-res. Data during flight 10. Support NovAtel GNSS board

Final Output CameraLink® IR camera

Electronic System

1.Integrate supplied components A.10MP Visual Band Camera B.1.3MP IR Camera C.Spatial Sensors i.NovAtel OEM Board OEMV3 ii.NovAtel OEM Board OEMV2 D.Camera Processing Board 2.Capture data from two cameras 3.Capture 1fps 4.Capture 30fps 5.Capture INS 30/sec (simultaneously) Spatial Sensing Customer Needs Met 6.External INS units 7.Data processing (overlay) 8.Real time viewing 9.Store full-res. Data during flight 10. Support NovAtel GNSS board

Output INS data format # of shot, (FLIGHT INFORMATION, Pitch, ect….) 675, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Electronic System

1.Integrate supplied components A.10MP Visual Band Camera B.1.3MP IR Camera C.Spatial Sensors i.NovAtel OEM Board OEMV3 ii.NovAtel OEM Board OEMV2 D.Camera Processing Board 2.Capture data from two cameras 3.Capture 1fps 4.Capture 30fps 5.Capture INS 30/sec (simultaneously) Processing elements Customer Needs Met 6.External INS units 7.Data processing (overlay) 8.Real time viewing 9.Store full-res. Data during flight 10. Support NovAtel GNSS board

OEM Digital Signal Processing Board Signal Processing already done on Customer Supplied Interface. – Image overlay – Compression – Resolution of Images Outputs – 10/100 Ethernet – S-Video Software interface available

OEM Board

Electronic System

FPGA Board Diagram

FPGA Board to Scale

1.Integrate supplied components A.10MP Visual Band Camera B.1.3MP IR Camera C.Spatial Sensors D.Camera Processing Board 2.Capture data from two cameras 3.Capture 1fps 4.Capture 30fps 5.Capture INS 30/sec (simultaneously) Processing elements Customer Needs Met 6.External INS units 7.Data processing (overlay) 8.Real time viewing 9.Store full-res. Data during flight 10. Support NovAtel GNSS board

FPGA Inputs/Outputs Flexible Architecture Faster Speed Parallel Processing DSP Energy Efficient Single Pipeline Easy Implementation Math based ISA Processing Elements

DSP Customer programmable – Encoding/Decoding media – Peripherals Role in this design – Image compression – Real time streaming of data – INS interface Required skills – Implementable Knowledge of C – DSP/BIOS

FPGA FPGA Selection – Quicker time to fabrication – Supreme configurability/Field reprogrammable – Has the I/O needed – Parallel processing

FPGA Xilinx Selection – Resources available to the team – Larger range of choices than other companies – Customer preference Model XC6SLX75T Selection – Package size (23mm x 23mm) – High speed transceiver count – I/O pin count – Cost effectiveness

FPGA Board Diagram

Data Flow – Initial Design Pictures Camera  FPGA  OEM INS Data INS  OEM

Data Flow – Final Design Pictures Camera  FPGA  OEM Camera  FPGA  HD INS Data INS  OEM  FPGA  HD

Data Speeds **Note: baud = bits per second (RS-232) Image – IR: 30 images / second VGA=640x MHz – Visible :1 image / second 10.7MP=3664x MHz INS – 30 captures / second 1kB=8kb 8000 baud

FPGA Pin Speeds Minimum values – 13ns -> 76 MHz – 5ns -> 200 MHz

System Software Design

FPGA Image Controller

Image Data Input

System Software Design

FPGA Central Dispatch

FPGA Process Flowchart Backup

FPGA Configurability Basis of configurability – Nature of transistor based FPGA Physical limitations – Through header on PCB using Xilinx provided development tools Backup

FPGA Configurability Customer configurable – Configuration languages Knowledge of VHDL/Verilog – Development packages Xilinx provided development tools – Physical configuration requirements Connect programmer and download data file, restart board Backup

Processing Elements DSP CPU based C Language FPGA Transistor based VHDL/Verilog Backup

Customer Needs Use Supplied Components – 10MP Visual Camera – IR Camera – 1 of the 2 Inertial Navigation Systems depending on availability NovAtel OEM Board OEMV3 NovAtel OEM Board OEMV2 – OEM Camera Processing Board Interface to single 10Mpixel Camera through proprietary “D3 Camera” connector. Interface to single Thermal Camera through Camera Link Interface. Capture 10MP data at 1FPS Capture the Thermal Camera data synchronized with the 10Mpixel camera. Capture INS data and store to match corresponding photos. Accept data from auxiliary external cameras and INS units Make data overlay and processing possible on-board Output data from the supplied OEM Board connection for real-time viewing Store data internally during flight using a SSD SATA drive. Package must include mounting and space necessary for four cameras. Package everything (except for the IR camera) to protect it against the environment and to minimize the size. “Everything” Includes: – (4) visual cameras and their lenses – (1) INS sensor – (1) OEM Camera Processing Board – Any other components necessary for operation Position images for ground observations Make cameras separable from the processing hardware Interface package to a light passenger aircraft.

Backup slide: Final Output IR Camera with INS Data overlay Color Aerial Photograph

Customer provided devices OEM Digital Signal Processing Board CameraLink® to D3 Conversion Chips Novatel OEM Board IMU System Cameras – (2) Using D3 Connector interface – (1) IR using Camera Link

Assembly