DWM: A Decomposable Winograd Method for Convolution Acceleration

  • Di Huang Chinese Academy of Sciences
  • Xishan Zhang Chinese Academy of Sciences
  • Rui Zhang Chinese Academy of Sciences
  • Tian Zhi Chinese Academy of Sciences
  • Deyuan He Chinese Academy of Sciences
  • Jiaming Guo Chinese Academy of Sciences
  • Chang Liu Chinese Academy of Sciences
  • Qi Guo Chinese Academy of Sciences
  • Zidong Du Chinese Academy of Sciences
  • Shaoli Liu Chinese Academy of Sciences
  • Tianshi Chen Chinese Academy of Sciences
  • Yunji Chen Chinese Academy of Sciences

Abstract

Winograd's minimal filtering algorithm has been widely used in Convolutional Neural Networks (CNNs) to reduce the number of multiplications for faster processing. However, it is only effective on convolutions with kernel size as 3x3 and stride as 1, because it suffers from significantly increased FLOPs and numerical accuracy problem for kernel size larger than 3x3 and fails on convolution with stride larger than 1. In this paper, we propose a novel Decomposable Winograd Method (DWM), which breaks through the limitation of original Winograd's minimal filtering algorithm to a wide and general convolutions. DWM decomposes kernels with large size or large stride to several small kernels with stride as 1 for further applying Winograd method, so that DWM can reduce the number of multiplications while keeping the numerical accuracy. It enables the fast exploring of larger kernel size and larger stride value in CNNs for high performance and accuracy and even the potential for new CNNs. Comparing against the original Winograd, the proposed DWM is able to support all kinds of convolutions with a speedup of ∼2, without affecting the numerical accuracy.

Published
2020-04-03
Section
AAAI Technical Track: Machine Learning