A WRM-based Application Daniele Felici – Università degli Studi di Roma “Tor Vergata” ER1 EDUSAFE Final Review CERN 20/06/2016 1

Summary Introduction Proposed solution Testing Campaign 2016 Hardware Measurements After the Testing Campaign What is in the future, me and the WRM

Introduction Roma 2 group proposed to study and use the previously presented Weighting Resistive Matrix (WRM) for new Augmented Reality (AR) applications

Introduction In the months before my enrolment: (Not yet at the time) Dr. Ali Abdallah studied and modelized the Matrix (see Ali presentation) An algorithm to recognize arbitrary-long segments was developed A full-software simulation was implemented and used in the EDUSAFE system, the segment recognition was good but the time performances were poor (~150 ms, Testing Campaign 2015) WRM Line Segments Reconstruction

Hardware development – Feasibility study WRM Line Segments Reconstruction ASIC (already developed in the past) To be developed and characterized To be developed and characterized For maximum performances a dedicated chip (ASIC) was the best solution The time was extremely short (less than 2 years to design, implementation, chip manufacturing, pcb design and tests) and the design was insidious => Go for a chip (ASIC) was risky and there were not controllable time delays (Chip manufacturing, PCB manufacturing)

Proposed Solution Before the 2016 Testing Campaign WRM Line Segments Reconstruction FPGA ASIC (already developed in the past) Software (Ali’s code) Use an FPGA (Xilinx Virtex-7): Pros: Low cost, flexibility, availability, fast development, fast debug, many possible interfaces, fast enough to process at least 30 FPS Cons: more power, lower performances (both negligible for a first prototype)

Proposed solution before the 2016 Testing Campaign Not ready before the testing campaign In the server: Long segment reconstruction and EPFL code interface Edges In the FPGA: Edge detection and Pcie interface Level-Adaptor Images WRM ADC 8-pixel segments 8-pixel segments PCIe

Testing Campaign Data Flow WRM FPGA FPGA Implementation in digital of the WRM equivalent algorithm Software

Testing Campaign 2016 Architecture In the server: Long segment reconstruction and EPFL code interface In the FPGA: Edge detection, WRM algorithm implementation and Pcie interface. The performances for EDUSAFE are ok. Most of the efforts are fundamental to integrate the real chip. Images 8-pixel segments PCIe

Timing test Pcie transmission time: ~7 ms Pcie transmission time: ~3.7 ms WRM FPGA FPGA implementation ~164 µs to have first pixel out of the FPGA-WRM and start the transmission ~3.9 ms to have the entire 640x480 image processed by the matrix Software ~5ms Logic working @ 80 MHz Pixels elaborated serially ~ 3.8 ms to have first pixel out of the edge detector and start the WRM operations Total execution time in the FPGA (edge detector + WRM) ~7.7 ms. Total execution time: ≤20ms Target ≤33,3 ms (30 FPS)

Performances (TC 2016) The difference is caused by a slightly difference in the Edge Detector implementation. However satisfying performance for EDUSAFE system   Lowest score Highest score Average LSD+Canny 14.5% 68.8% 44.25% LSD 10.6% 68.3% 39.4% Hough 1.3% 54.7% 25.3% WRM SW 16.4% 75.8% 46.7% WRM FPGA 9.2% 64% 40.0% TC-2015 FPGA TC-2016 SW

After the testing campaign Canny data processing precision improved from 8 to 32 bit. This brings the performances of the HW closer to the SW with unchanged execution time Implemented a frame sharpness scoring system in the FPGA. This helps the EPFL algorithm to choose the best frames to process increasing the general pose calculation performances Real WRM integration started

What’s next - Me I received and accepted a very good job offer from a Swiss Company as R&D engineer – FPGA designer I started from May 2nd, 2016 Unfortunately, this short advice stopped me in developing WRM and prevent me to stay with you in person All the expertise (hard and soft skills) developed during EDUSAFE has been fundamental to get the new job

What’s next - WRM Few ideas: Real WRM integration (It was ongoing) FPGA<->PC transmission optimization Develop a new generation of WRM The FPGA implementation can be used to test new ideas or new (or really old) applications

for all.