WP2 – Testing campaign and beyond G.Aielli for the WP2 CERN Technical Meeting 28/01/2016 WP2 – Testing campaign and beyond
xPCI Frame grabber/ FPGA Setup for the testing campaign HDMI USB3 Arrow Tx Visual display PTU Real time Wireless Full HD image Display driver Animation engine IMU Arrow RX IMU data WiFi Pose server xPCI Frame grabber/ FPGA Computer memory WRM prototype Visual tracking Sensor fusion
Status in the final testing campaign Impacting new developments HW chain streamlining No branching path for video No overload of the PTU by concentrating the pose calculation in the server WRM in the system I/O interface definition All frames are analyzed and scored for “quality” WRM emulated on FPGA with comparable performance Final implementation not ready for the FTC will be finished in the next few months Filter segments based on part detection IMU real time stream to the server: To be completed (just after the FTC) New implementation of visual algorithm: IMU data used for better prior determination Minimization done on both key features and segments Estimate the pose form incremental segments Streamlining of the tracking software Can choose the next frame to process basing on the “best” in the buffer
Strategy proposed in Athens Increment the statistical reliability of the tracking by means of a complementarity approach: Each of the techniques can benefit from partial inputs of the others, in particular of the faster on the slower: IMUWRMVisual Tracking Object detection and camera pose determination are not necessarily coincident, being the latter needed more frequently than the former Where the pose determination is poor or delayed with a technique, there is a probability to succeed with another one e.g.: Interpolating the VT with the IMU or with the incremental segments Use the segments to make a trained regression directly in the parametric space
Focus 1 – fast camera and image transmission What if we are able to pre process the frames in real time, whatever the frame rate? Use a faster camera sharper frames Buffer the frames Score the frames The slower processes such as the object detection will compute only the best scored ones The others used as incremental segment pose interpolation Software framework to be modified adding a buffer shortly after TC server xPCI Frame grabber/ FPGA Computer memory WRM prototype Visual tracking Sensor fusion server xPCI Frame grabber/ FPGA Computer memory 30 fps WRM prototype 4 fps Visual tracking Sensor fusion s1 s2 s3 s4 s5 f1 f2 f3 f4 f5
Focus-2 WRM final implementation Due to TC deadline and delays in the readout development we emulated the WRM on FPGA at low level. No substantial performance limitation by exploiting a very smart WRM approximation (valid only for this simple and semi digital WRM version) In this very special case a top level FPGA can barely compete with a 20 years old WRM in a minimal implementation and with a >> power consumption It is a perfect test bed We will complete the WRM test system in Roma ASAP The FPGA based emulator will stay at CERN to permit further testing
Focus 3 - improved tracking Implement as much as possible the Athens meeting strategy: Tracking algorithm served on request with the best frame and segment set in the buffer See the effect of using a faster camera within the max. WRM speed (by now >80 fps) Early interpolated pose boosted by IMU predictions Early interpolated pose boosted by incremental segments from WRM Study a soft trained regression algorithm, using the parametric representation of the segments and of the searched object, to determine a pose, as a complement to the corner detection
Next developments (for future applications) Testing Campaign is just a checkpoint of the current progress, we expect to continue just after: The point addressed in the focuses can be still considered part of EDUSAFE Next the WRM setup will be made independent from the server toward a stand alone miniaturized component setup ASIC will replace the FPGA (mid term) New more efficient WRM chip (mid/long term) What above is BEYOND EDUSAFE and needs relevant extra funds…