SKELETON-BASED ACTION RECOGNITION USING FEATURE FUSION FOR SPATIAL-TEMPORAL GRAPH CONVOLUTIONAL NETWORKS

  • Dinh Tan Pham Faculty of Information Technology, Hanoi University of Mining and Geology, Hanoi, Vietnam; Computer Vision Department, MICA Institute, Hanoi University of Science and Technology, Hanoi, Vietnam
  • Thi Phuong Dang School of Electronics and Telecommunications, Hanoi University of Science and Technology, Hanoi, Vietnam
  • Duc Quang Nguyen School of Electronics and Telecommunications, Hanoi University of Science and Technology, Hanoi, Vietnam
  • Thi Lan Le School of Electronics and Telecommunications, Hanoi University of Science and Technology, Hanoi, Vietnam; Computer Vision Department, MICA Institute, Hanoi University of Science and Technology, Hanoi, Vietnam
  • Hai Vu School of Electronics and Telecommunications, Hanoi University of Science and Technology, Hanoi, Vietnam; Computer Vision Department, MICA Institute, Hanoi University of Science and Technology, Hanoi, Vietnam
Keywords: Action recognition, graph convolutional network, skeletal data, feature fusion

Abstract

Human action recognition (HAR) has been used in a variety of applications such as gaming, healthcare, surveillance, and robotics. Research on utilizing data such as color, depth, and skeletal data has been extensively conducted to achieve high-performance HAR. Compared with color and depth data, skeletal data are more compact, therefore, they are more efficient for computation and storage. Moreover, skeletal data are invariant to clothing textures, background, and lighting conditions. With the booming of deep learning, HAR has received a lot of attention. Spatial-Temporal Graph Convolution Networks (ST-GCN) have proved to be state-of-the-art architecture for HAR using skeleton data. However, this does not hold when working with challenging datasets that contain incomplete and noisy skeletal data. In this paper, a new method is proposed for HAR by adding a Feature Fusion module and applying hyperparameter optimization. The performance of the proposed method is evaluated on the challenging dataset CMDFALL and the newly-built MICA-Action3D dataset. Experimental results show that the proposed method significantly improves the performance of ST-GCN on these challenging datasets.

điểm /   đánh giá
Published
2021-10-01
Issue
Vol. 10 No. 01 (2021)
Section
Bài viết