Temporally smooth online action detection using cycle-consistent future anticipation

Young Hwi Kim, Seonghyeon Nam, Seon Joo Kim

Research output: Contribution to journalArticlepeer-review

11 Citations (Scopus)


Many video understanding tasks work in the offline setting by assuming that the input video is given from the start to the end. However, many real-world problems require the online setting, making a decision immediately using only the current and the past frames of videos such as in autonomous driving and surveillance systems. In this paper, we present a novel solution for online action detection by using a simple yet effective RNN-based networks called the Future Anticipation and Temporally Smoothing network (FATSnet). The proposed network consists of a module for anticipating the future that can be trained in an unsupervised manner with the cycle-consistency loss, and another component for aggregating the past and the future for temporally smooth frame-by-frame predictions. We also propose a solution to relieve the performance loss when running RNN-based models on very long sequences. Evaluations on TVSeries, THUMOS'14, and BBDB show that our method achieve the state-of-the-art performances compared to the previous works on online action detection.

Original languageEnglish
Article number107954
JournalPattern Recognition
Publication statusPublished - 2021 Aug

Bibliographical note

Publisher Copyright:
© 2021 Elsevier Ltd

All Science Journal Classification (ASJC) codes

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence


Dive into the research topics of 'Temporally smooth online action detection using cycle-consistent future anticipation'. Together they form a unique fingerprint.

Cite this