Main > Laser & Optoelectronics Progress >  Volume 57 >  Issue 20 >  Page 201506 > Article
  • Abstract
  • Abstract
  • View Summary
  • Figures (10)
  • Tables (6)
  • Equations (0)
  • References (31)
  • Get PDF(in Chinese)
  • Paper Information
  • Received: Dec. 23, 2019

    Accepted: Feb. 25, 2020

    Posted: Oct. 1, 2020

    Published Online: Oct. 17, 2020

    The Author Email: Kong Jun (kongjun@jiangnan.edu.cn)

    DOI: 10.3788/LOP57.201506

  • Get Citation
  • Copy Citation Text

    Fuzheng Guo, Jun Kong, Min Jiang. Action Recognition Based on Adaptive Fusion of RGB and Skeleton Features[J]. Laser & Optoelectronics Progress, 2020, 57(20): 201506

    Download Citation

  • Category
  • Machine Vision
  • Share
Laser & Optoelectronics Progress, Vol. 57, Issue 20, 201506 (2020)

Action Recognition Based on Adaptive Fusion of RGB and Skeleton Features

Guo Fuzheng, Kong Jun*, and Jiang Min

Author Affiliations

  • International Joint Laboratory for Pattern Recognition and Computational Intelligence, Jiangnan University, Wuxi, Jiangsu 214122, China

Abstract

In this paper, we proposed an action recognition algorithm based on the adaptive fusion of RGB and skeleton features to efficiently improve the accuracy of action recognition. However, the conventional action recognition algorithms based on RGB and skeleton features generally suffer from the inability to effectively utilize the complementarity of the two features and thus fail to focus on important frames in the video. Considering this, we first used the bidirectional long short-term memory network (Bi-LSTM) combined with a self-attention mechanism to extract spatial-temporal features of RGB and skeleton images. Next, we constructed an adaptive weight computing network (AWCN) and computed these adaptive weights based on the spatial features of two types of images. Finally, the extracted spatial-temporal features were fused by the adaptive weights to implement action recognition. Using Penn Action, JHMDB, and NTU RGB-D dataset, the experimental results show that our proposed algorithm effectively improves the accuracy of action recognition compared with existing methods.

keywords

Please Enter Your Email: