Super Sparse Convolutional Neural Networks

Authors

  • Yao Lu Harbin Institute of Technology
  • Guangming Lu Harbin Institute of Technology
  • Bob Zhang University of Macau
  • Yuanrong Xu Harbin Institute of Technology
  • Jinxing Li Hong Kong Polytechnic University

DOI:

https://doi.org/10.1609/aaai.v33i01.33014440

Abstract

To construct small mobile networks without performance loss and address the over-fitting issues caused by the less abundant training datasets, this paper proposes a novel super sparse convolutional (SSC) kernel, and its corresponding network is called SSC-Net. In a SSC kernel, every spatial kernel has only one non-zero parameter and these non-zero spatial positions are all different. The SSC kernel can effectively select the pixels from the feature maps according to its non-zero positions and perform on them. Therefore, SSC can preserve the general characteristics of the geometric and the channels’ differences, resulting in preserving the quality of the retrieved features and meeting the general accuracy requirements. Furthermore, SSC can be entirely implemented by the “shift” and “group point-wise” convolutional operations without any spatial kernels (e.g., “3×3”). Therefore, SSC is the first method to remove the parameters’ redundancy from the both spatial extent and the channel extent, leading to largely decreasing the parameters and Flops as well as further reducing the img2col and col2img operations implemented by the low leveled libraries. Meanwhile, SSC-Net can improve the sparsity and overcome the over-fitting more effectively than the other mobile networks. Comparative experiments were performed on the less abundant CIFAR and low resolution ImageNet datasets. The results showed that the SSC-Nets can significantly decrease the parameters and the computational Flops without any performance losses. Additionally, it can also improve the ability of addressing the over-fitting problem on the more challenging less abundant datasets.

Downloads

Published

2019-07-17

How to Cite

Lu, Y., Lu, G., Zhang, B., Xu, Y., & Li, J. (2019). Super Sparse Convolutional Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 4440-4447. https://doi.org/10.1609/aaai.v33i01.33014440

Issue

Section

AAAI Technical Track: Machine Learning