Image Cropping with Composition and Saliency Aware Aesthetic Score Map

  • Yi Tu Shanghai Jiao Tong University
  • Li Niu Shanghai Jiao Tong University
  • Weijie Zhao Versa Inc
  • Dawei Cheng Shanghai Jiao Tong University
  • Liqing Zhang Shanghai Jiao Tong University

Abstract

Aesthetic image cropping is a practical but challenging task which aims at finding the best crops with the highest aesthetic quality in an image. Recently, many deep learning methods have been proposed to address this problem, but they did not reveal the intrinsic mechanism of aesthetic evaluation. In this paper, we propose an interpretable image cropping model to unveil the mystery. For each image, we use a fully convolutional network to produce an aesthetic score map, which is shared among all candidate crops during crop-level aesthetic evaluation. Then, we require the aesthetic score map to be both composition-aware and saliency-aware. In particular, the same region is assigned with different aesthetic scores based on its relative positions in different crops. Moreover, a visually salient region is supposed to have more sensitive aesthetic scores so that our network can learn to place salient objects at more proper positions. Such an aesthetic score map can be used to localize aesthetically important regions in an image, which sheds light on the composition rules learned by our model. We show the competitive performance of our model in the image cropping task on several benchmark datasets, and also demonstrate its generality in real-world applications.

Published
2020-04-03
Section
AAAI Technical Track: Vision