Fast Learning of Temporal Action Proposal via Dense Boundary Generator

  • Chuming Lin Tencent
  • Jian Li Tencent
  • Yabiao Wang Tencent
  • Ying Tai Tencent
  • Donghao Luo Tencent
  • Zhipeng Cui Tencent
  • Chengjie Wang Tencent
  • Jilin Li Tencent
  • Feiyue Huang Tencent
  • Rongrong Ji Xiamen University


Generating temporal action proposals remains a very challenging problem, where the main issue lies in predicting precise temporal proposal boundaries and reliable action confidence in long and untrimmed real-world videos. In this paper, we propose an efficient and unified framework to generate temporal action proposals named Dense Boundary Generator (DBG), which draws inspiration from boundary-sensitive methods and implements boundary classification and action completeness regression for densely distributed proposals. In particular, the DBG consists of two modules: Temporal boundary classification (TBC) and Action-aware completeness regression (ACR). The TBC aims to provide two temporal boundary confidence maps by low-level two-stream features, while the ACR is designed to generate an action completeness score map by high-level action-aware features. Moreover, we introduce a dual stream BaseNet (DSB) to encode RGB and optical flow information, which helps to capture discriminative boundary and actionness features. Extensive experiments on popular benchmarks ActivityNet-1.3 and THUMOS14 demonstrate the superiority of DBG over the state-of-the-art proposal generator (e.g., MGG and BMN).

AAAI Technical Track: Vision