Rethinking Temporal Fusion for Video-Based Person Re-Identification on Semantic and Time Aspect

  • Xinyang Jiang Tencent
  • Yifei Gong Tencent
  • Xiaowei Guo Tencent
  • Qize Yang Sun Yat-sen University
  • Feiyue Huang Tencent
  • WEI-SHI ZHENG Sun Yat-sen University
  • Feng Zheng Southern University of Science and Technology
  • Xing Sun Tencent

Abstract

Recently, the research interest of person re-identification (ReID) has gradually turned to video-based methods, which acquire a person representation by aggregating frame features of an entire video. However, existing video-based ReID methods do not consider the semantic difference brought by the outputs of different network stages, which potentially compromises the information richness of the person features. Furthermore, traditional methods ignore important relationship among frames, which causes information redundancy in fusion along the time axis. To address these issues, we propose a novel general temporal fusion framework to aggregate frame features on both semantic aspect and time aspect. As for the semantic aspect, a multi-stage fusion network is explored to fuse richer frame features at multiple semantic levels, which can effectively reduce the information loss caused by the traditional single-stage fusion. While, for the time axis, the existing intra-frame attention method is improved by adding a novel inter-frame attention module, which effectively reduces the information redundancy in temporal fusion by taking the relationship among frames into consideration. The experimental results show that our approach can effectively improve the video-based re-identification accuracy, achieving the state-of-the-art performance.

Published
2020-04-03
Section
AAAI Technical Track: Vision