1. RIT Senior Design Project 10662 D3 Engineering Camera Platform Friday November 6, 2009 9:00am to 11:00am

2. Team Members Gregory Hintz (EE) – Project Manager Samuel Skalicky (CE) – Lead Engineer, FPGA Board Jeremy Greene (EE) – Connector Board Jared Burdick (EE) – Power Michelle Bard (ME) – Environmental Tony Perrone (ME) – Physical Design

3. Advisors Scott Reardon (D3 Engineering) Kevin Kearney (D3 Engineering) Dr. Robert Kremens (RIT-Imaging Science) Philip Bryan (RIT – Industry Guide)

4. Project Status Risks BOM Analysis Feasibility Designs Test Plans

5. Schedule for the Design Review Overview – Gregory Hintz Electrical Discussion Processor Board and FPGA – Samuel Skalicky Connector Board, INS System – Jeremy Greene Mechanical Discussion System Design – Tony Perrone Environmental Concerns – Michelle Bard

6. What is the Customer Looking for? Integrate supplied components Ruggedized Unit Flight-capable package Can record and transmit Capable of processing

7. Black Box System Model

8. 1.Integrate supplied components A.10MP Visual Band Camera B.1.3MP IR Camera C.Spatial Sensors i.NovAtel OEM Board OEMV3 ii.NovAtel OEM Board OEMV2 D.Camera Processing Board 2.Capture data from two cameras 3.Capture 10MP @ 1fps 4.Capture 1.3MP @ 30fps 5.Capture INS data @ 30/sec (simultaneously) Camera Components Customer Needs Met 6.External INS units 7.Data processing (overlay) 8.Real time viewing 9.Store full-res. Data during flight 10. Support NovAtel GNSS board

9. Electronic System

10. 1.Integrate supplied components A.10MP Visual Band Camera B.1.3MP IR Camera C.Spatial Sensors i.NovAtel OEM Board OEMV3 ii.NovAtel OEM Board OEMV2 D.Camera Processing Board 2.Capture data from two cameras 3.Capture 10MP @ 1fps 4.Capture 1.3MP @ 30fps 5.Capture INS data @ 30/sec (simultaneously) Camera Components Customer Needs Met 6.External INS units 7.Data processing (overlay) 8.Real time viewing 9.Store full-res. Data during flight 10. Support NovAtel GNSS board

11. Final Output 10mp Camera

12. Electronic System

13. 1.Integrate supplied components A.10MP Visual Band Camera B.1.3MP IR Camera C.Spatial Sensors i.NovAtel OEM Board OEMV3 ii.NovAtel OEM Board OEMV2 D.Camera Processing Board 2.Capture data from two cameras 3.Capture 10MP @ 1fps 4.Capture 1.3MP @ 30fps 5.Capture INS data @ 30/sec (simultaneously) Camera Components Customer Needs Met 6.External INS units 7.Data processing (overlay) 8.Real time viewing 9.Store full-res. Data during flight 10. Support NovAtel GNSS board

14. Final Output CameraLink® IR camera

15. Electronic System

16. 1.Integrate supplied components A.10MP Visual Band Camera B.1.3MP IR Camera C.Spatial Sensors i.NovAtel OEM Board OEMV3 ii.NovAtel OEM Board OEMV2 D.Camera Processing Board 2.Capture data from two cameras 3.Capture 10MP @ 1fps 4.Capture 1.3MP @ 30fps 5.Capture INS data @ 30/sec (simultaneously) Spatial Sensing Customer Needs Met 6.External INS units 7.Data processing (overlay) 8.Real time viewing 9.Store full-res. Data during flight 10. Support NovAtel GNSS board

17. Output INS data format # of shot, (FLIGHT INFORMATION, Pitch, ect….) 675,495060.000000,39.377116,-82.774721,1442.237213,177.966706,-6.573488,87.156026 676,495063.000000,39.375698,-82.773364,1437.212509,178.655193,-2.978762,89.591399 677,495066.000000,39.374288,-82.771967,1432.054779,177.896426,-3.434334,92.544970 678,495069.000000,39.372892,-82.770509,1428.557648,177.391126,-12.302477,92.517868 679,495072.000000,39.371514,-82.768882,1425.166306,178.138512,-9.035039,88.154010 680,495075.000000,39.370128,-82.767133,1425.035141,176.875920,-1.502685,89.783104 681,495078.000000,39.368761,-82.765407,1424.326828,176.056416,-6.737449,90.066296 682,495081.000000,39.367471,-82.763622,1421.768311,175.569431,-7.973052,88.809422 683,495084.000000,39.366537,-82.761748,1427.402252,176.045872,-7.985133,86.404080

18. Electronic System

19. 1.Integrate supplied components A.10MP Visual Band Camera B.1.3MP IR Camera C.Spatial Sensors i.NovAtel OEM Board OEMV3 ii.NovAtel OEM Board OEMV2 D.Camera Processing Board 2.Capture data from two cameras 3.Capture 10MP @ 1fps 4.Capture 1.3MP @ 30fps 5.Capture INS data @ 30/sec (simultaneously) Processing elements Customer Needs Met 6.External INS units 7.Data processing (overlay) 8.Real time viewing 9.Store full-res. Data during flight 10. Support NovAtel GNSS board

20. OEM Digital Signal Processing Board Signal Processing already done on Customer Supplied Interface. – Image overlay – Compression – Resolution of Images Outputs – 10/100 Ethernet – S-Video Software interface available

21. OEM Board

22. Electronic System

23. FPGA Board Diagram

24. FPGA Board to Scale

25. 1.Integrate supplied components A.10MP Visual Band Camera B.1.3MP IR Camera C.Spatial Sensors D.Camera Processing Board 2.Capture data from two cameras 3.Capture 10MP @ 1fps 4.Capture 1.3MP @ 30fps 5.Capture INS data @ 30/sec (simultaneously) Processing elements Customer Needs Met 6.External INS units 7.Data processing (overlay) 8.Real time viewing 9.Store full-res. Data during flight 10. Support NovAtel GNSS board

26. FPGA Inputs/Outputs Flexible Architecture Faster Speed Parallel Processing DSP Energy Efficient Single Pipeline Easy Implementation Math based ISA Processing Elements

27. DSP Customer programmable – Encoding/Decoding media – Peripherals Role in this design – Image compression – Real time streaming of data – INS interface Required skills – Implementable Knowledge of C – DSP/BIOS

28. FPGA FPGA Selection – Quicker time to fabrication – Supreme configurability/Field reprogrammable – Has the I/O needed – Parallel processing

29. FPGA Xilinx Selection – Resources available to the team – Larger range of choices than other companies – Customer preference Model XC6SLX75T Selection – Package size (23mm x 23mm) – High speed transceiver count – I/O pin count – Cost effectiveness

30. FPGA Board Diagram

31. Data Flow – Initial Design Pictures Camera  FPGA  OEM INS Data INS  OEM

32. Data Flow – Final Design Pictures Camera  FPGA  OEM Camera  FPGA  HD INS Data INS  OEM  FPGA  HD

33. Data Speeds **Note: baud = bits per second (RS-232) Image – IR: 30 images / second VGA=640x480 9.2 MHz – Visible :1 image / second 10.7MP=3664x2748 10.07 MHz INS – 30 captures / second 1kB=8kb 8000 baud

34. FPGA Pin Speeds Minimum values – 13ns -> 76 MHz – 5ns -> 200 MHz

35. System Software Design

36. FPGA Image Controller

37. Image Data Input

38. System Software Design

39. FPGA Central Dispatch

40. FPGA Process Flowchart Backup

41. FPGA Configurability Basis of configurability – Nature of transistor based FPGA Physical limitations – Through header on PCB using Xilinx provided development tools Backup

42. FPGA Configurability Customer configurable – Configuration languages Knowledge of VHDL/Verilog – Development packages Xilinx provided development tools – Physical configuration requirements Connect programmer and download data file, restart board Backup

43. Processing Elements DSP CPU based C Language FPGA Transistor based VHDL/Verilog Backup

44. Customer Needs Use Supplied Components – 10MP Visual Camera – IR Camera – 1 of the 2 Inertial Navigation Systems depending on availability NovAtel OEM Board OEMV3 NovAtel OEM Board OEMV2 – OEM Camera Processing Board Interface to single 10Mpixel Camera through proprietary “D3 Camera” connector. Interface to single Thermal Camera through Camera Link Interface. Capture 10MP data at 1FPS Capture the Thermal Camera data synchronized with the 10Mpixel camera. Capture INS data and store to match corresponding photos. Accept data from auxiliary external cameras and INS units Make data overlay and processing possible on-board Output data from the supplied OEM Board connection for real-time viewing Store data internally during flight using a SSD SATA drive. Package must include mounting and space necessary for four cameras. Package everything (except for the IR camera) to protect it against the environment and to minimize the size. “Everything” Includes: – (4) visual cameras and their lenses – (1) INS sensor – (1) OEM Camera Processing Board – Any other components necessary for operation Position images for ground observations Make cameras separable from the processing hardware Interface package to a light passenger aircraft.

45. Backup slide: Final Output IR Camera with INS Data overlay Color Aerial Photograph

46. Customer provided devices OEM Digital Signal Processing Board CameraLink® to D3 Conversion Chips Novatel OEM Board IMU System Cameras – (2) Using D3 Connector interface – (1) IR using Camera Link

47. Assembly