Download presentation
Presentation is loading. Please wait.
1
Embedded OpenCV Acceleration
Dario Pennisi
2
Open-Source Computer Vision Library Over 2500 algorithms and functions
Introduction Open-Source Computer Vision Library Over 2500 algorithms and functions Cross platform, portable API Windows, Linux, OS X, Android, iOS Real Time performance BSD license Professionally developed and maintained 19/09/2018
3
History Launched in 1999 by Intel First Alpha released in 2000
Showcasing Intel Performance Library First Alpha released in 2000 1.0 version released in 2006 Corporate support by Willow Garage in 2008 2.0 version released in 2009 Improved c++ interfaces Releases each 6 months In 2014 taken over by ItSeez 3.0 in beta now Drop C API support 19/09/2018
4
Application structure
Building blocks to ease vision applications Image Retrieval Pre Processing Feature Extraction Object Detection OpenCV highgui imgproc features2d objdetect calib3d video stitching ml Recognition Reconstruction Analisys Decision Making 19/09/2018
5
Environment 19/09/2018 Application C++ Java Python OpenCV
cv::parallel_for_ Threading APIs Concurrency CStripes GCD OpenMP TBB OS Acceleration SSE/AVX/NEON OpenCL CUDA 19/09/2018
6
Desktop vs Embedded Desktop Industrial Embedded Cores/Threads 8/16 4/4
Core Frequency >4GHz >1.4GHz L1 Cache 32K+32K L2 Cache 256K per core 2M shared L3 Cache 20M - DDR Controllers 4x64 Bit 1066 MHz 2x32 Bit 800MHz TDP 140W (CPU) 10W (SoC) GPU cores 2880 1+4+16 19/09/2018
7
Dimensioning system is fundamental
System Engineering Dimensioning system is fundamental Understand your algorithm Carefully choose your toolbox Embedded means no chance for “one size fits all” 19/09/2018
8
Acceleration Strategies
Optimize Algorithms Profile Optimize Partition (CPU/GPU/DSP) FPGA acceleration High level synthesis Custom DSP RTL coding Brute Force Increase number of CPUs Increase CPU Frequency Accelerated libraries NEON OpenCL/CUDA 19/09/2018
9
Bottlenecks Know your enemy 19/09/2018
10
Access to external memory is expensive
CPU load instructions are slow Memory has Latency Memory bandwidth is shared among CPUs Cache Prevents CPU to access external memory Data and instruction 19/09/2018
11
What happens when we have cache miss?
Disordered accesses What happens when we have cache miss? Fetch data from same memory row 13 clocks Fetch data from a different row 23 clocks Cache line usually 32 bytes 8 clocks to fill a line (32 bit data bus) Memory bandwidth Efficiency 38% on same row 26% on different row 19/09/2018
12
Bottlenecks - Cache 1920x1080 YCbCr 4:2:2 (Full HD) 4MB
Double the size of the biggest ARM L2 cache 1280x720 YCbCr 4:2:2 (HD) 1.8 MB Just fits L2 Cache… ok if reading and writing to the same frame 720x576 YCbCr 4:2:2 (SD) 800KB 2 images in L2 cache… 19/09/2018
13
Mostly designed for PCs
OpenCV Algorithms Mostly designed for PCs Well structured General purpose Optimized functions for SSE/AVX Relatively optimized Small number of accelerated functions NEON Cuda (nVidia GPU/Tegra) OpenCL (GPU, Multicore processors) 19/09/2018
14
NEON SIMD instructions work on vectors of registers
Multicore ARM/NEON NEON SIMD instructions work on vectors of registers Load-process-store philosophy Load/store costs 1 cycle only if in L1 cache 4-12 cycles if in L2 25 to 35 cycles on L2 cache miss SIMD instructions can take from 1 to 5 clocks Fast clock useless on big datasets/small computation 19/09/2018
15
Very similar to ARM/NEON
Generic DSP Very similar to ARM/NEON High speed pipeline impaired by inefficient memory access subsystem When smart DMA is available it is very complex to program When DSP is integrated in SoC it shares ARM’s bandwidth 19/09/2018
16
Bandwidth and inefficiencies:
OpenCL on GPU OpenCL on Vivante GC2000 Claimed capability up to 16 GFLOPS Real Applications only on internal registers: 13.8 GFLOPS computing 1000x1000 matrix: 600 MFLOPS Bandwidth and inefficiencies: Only 1K local memory and 64 byte memory cache 19/09/2018
17
Same code can run on FPGA and GPU
OpenCL on FPGA Same code can run on FPGA and GPU Transform selected functions in hardware Automated memory access coalescing Each function requires dedicated logic Large FPGAs required Partial reconfiguration may solve this Significant compilation time 19/09/2018
18
HLS requires Code to be heavily modified
HLS on FPGA High Level Synthesis Convert C to hardware HLS requires Code to be heavily modified Pragmas to instruct compiler Code restructuring Not portable anymore Each function requires dedicated logic Large FPGAs required Partial reconfiguration may solve this Significant compilation time 19/09/2018
19
Data intensive processing
A different approach Demanding algorithms on low cost/power HW Algorithm Analysis Memory Access Pattern Data intensive processing Decision Making DMA DSP NEON ARM program Custom Instruction (RTL) 19/09/2018
20
External co-processing
ARM GPU Memory FPGA PCIe ARM Memory FPGA 19/09/2018
21
Co-processor details FPGA Co-Processor Separate memory
Adds bandwidth Reduces access conflict Algorithm aware DMA Access memory in ordered way Add caching through embedded RAM Algorithm specific processors HLS/OpenCL synthesized IP blocks DSP with custom instructions Hardcoded IP blocks Block capture DPRAM(s) DSP core (s) Memory DMA Processor DPRAM DSP core/IP Block ARM 19/09/2018
22
Co-processor details Flex DMA Block Capture DPRAM DSP Core
Dedicated processor with DMA custom instruction Software defined memory access pattern Block Capture Extracts data for each tile DPRAM Local, high speed cache DSP Core Dedicated processor with Algorithm specific custom instructions ARM ARM Memory Flex DMA Flex DMA Flex DMA Flex DMA Block capture Block capture Block capture Block capture Block capture Block capture DPRAM(s) DPRAM(s) DPRAM(s) DPRAM(s) DPRAM(s) DPRAM(s) DPRAM DPRAM(s) DPRAM DPRAM(s) DPRAM DPRAM DSP core (s) DSP core (s) DSP core (s) DSP core (s) DSP core/IP Block DSP core/IP Block 19/09/2018
23
Environment 19/09/2018 Application C++ Java Python OpenCV
cv::parallel_for_ Threading APIs Concurrency CStripes GCD OpenMP TBB OS Acceleration SSE/AVX/NEON OpenCL CUDA FPGA OpenVX 19/09/2018
24
OpenVX 19/09/2018
25
OpenVX Graph Manager Graph Construction Graph Execution
Allocates resources Logical representation of algorithm Graph Execution Concatenate nodes avoiding memory storage Tiling extensions Single node execution can be split in multiple tiles Multiple accelerators executing single task in parallel Memory Node2 Node1 Node2 Memory Node1 19/09/2018
26
Summary What we learnt OpenCV today is mainly PC oriented.
ARM, Cuda, OpenCL support growing Existing acceleration only on selected functions Embedded CV requires good partitioning among resources When ASSPs are not enough FPGAs are key OpenVX provides a consistent HW acceleration platform, not only for OpenCV What we learnt 19/09/2018
27
Questions 19/09/2018
28
Thank you 19/09/2018
Similar presentations
