Image Formation Model Guided Deep Image Super-Resolution

  • Jinshan Pan Nanjing University of Science and Technology
  • Yang Liu Dalian University of Technology
  • Deqing Sun Google
  • Jimmy Ren SenseTime Research
  • Ming-Ming Cheng Nankai University
  • Jian Yang Nanjing University of Science and Technology
  • Jinhui Tang Nanjing University of Science and Technology


We present a simple and effective image super-resolution algorithm that imposes an image formation constraint on the deep neural networks via pixel substitution. The proposed algorithm first uses a deep neural network to estimate intermediate high-resolution images, blurs the intermediate images using known blur kernels, and then substitutes values of the pixels at the un-decimated positions with those of the corresponding pixels from the low-resolution images. The output of the pixel substitution process strictly satisfies the image formation model and is further refined by the same deep neural network in a cascaded manner. The proposed framework is trained in an end-to-end fashion and can work with existing feed-forward deep neural networks for super-resolution and converges fast in practice. Extensive experimental results show that the proposed algorithm performs favorably against state-of-the-art methods.

AAAI Technical Track: Vision