Research Behavior Analysis & Understanding

Robust Human Action Recognition with Considering Action Transition and Posture Features Extracted by Using Convolutional Neural Networks

In this research, we propose a behavior recognition method that is robust to changes in the shooting environment and resistant to time shift during behavior transit by learning with attitude information abstracted in higher dimensions as input, taking into consideration the continuity of behavior. Furthermore, we propose a method to detect action transition frames using output values ​​of each class at the time of behavior recognition, and improve behavior recognition accuracy during state transition by supplementing behaviors between transition frames. In the proposed method, we first create a posture estimator that learns the joint position using Convolutional Neural Networks (CNN), and create an attitude feature extractor from the intermediate structure of the created estimator. Next, learning behavior by Long Short-Term Memory (LSTM) with the posture feature obtained from the extractor as input.

Action detection of TV viewer by object information and image feature