1. Instruction Driven Cross-Layer CNN Accelerator with Winograd Transformation on FPGA Jincheng Yu, Yiming Hu, Xuefei Ning, Jiantao Qiu, Kaiyuan Guo, Yu Wang, Huazhong Yang Dept. E.E., Tsinghua University, Beijing, China
Key problems of CNN accelerator on FPGA
Memory Access
Flexibility
Peak Performance
Solutions for each problem
Cross-Layer Scheduling
Instruction Set
Winograd Transformation
FPGA is adopted to accelerate CNN due to its high performance, high energy efficiency, and flexibility.
Memory Access dominates the energy consumption of CNN accelerators rather than computation units. Cross layer scheduling policy can minimize the data intermediate data transfer by using on-chip memory instead of off-chip memory to cache the intermediate data between different layers
Flexibility is important for the hardware accelerator because the great variety of the topologies of state-of-the-art CNNs brings challenge to hardware. An Instruction set is can drive different CNN on the same hardware.
Winograd transformation can leverage FPGA and improve the peak performance, since several times of multiplication can be done with the same hardware resources
In our work, a CNN is divided into several layer blobs to minimize data transfer. The compiler of our instruction set transfer each layer blob into instructions with cross-layer and Winograd. We also design a hardware to run the instructions.
Workflow
Network dividing to minimize data transfer
Translate CNN into instructions
Run instructions on FPGA