Instruction Driven Cross-Layer CNN Accelerator with Winograd Transformation on FPGA Jincheng Yu, Yiming Hu, Xuefei Ning, Jiantao Qiu, Kaiyuan Guo, Yu Wang, Huazhong Yang Dept. E.E., Tsinghua University, Beijing, China Key problems of CNN accelerator on FPGA Memory Access Flexibility Peak Performance Solutions for each problem Cross-Layer Scheduling Instruction Set Winograd Transformation FPGA is adopted to accelerate CNN due to its high performance, high energy efficiency, and flexibility. Memory Access dominates the energy consumption of CNN accelerators rather than computation units. Cross layer scheduling policy can minimize the data intermediate data transfer by using on-chip memory instead of off-chip memory to cache the intermediate data between different layers Flexibility is important for the hardware accelerator because the great variety of the topologies of state-of-the-art CNNs brings challenge to hardware. An Instruction set is can drive different CNN on the same hardware. Winograd transformation can leverage FPGA and improve the peak performance, since several times of multiplication can be done with the same hardware resources In our work, a CNN is divided into several layer blobs to minimize data transfer. The compiler of our instruction set transfer each layer blob into instructions with cross-layer and Winograd. We also design a hardware to run the instructions. Workflow Network dividing to minimize data transfer Translate CNN into instructions Run instructions on FPGA