Exploring Human Action Recognition for Rehabilitation Game Application

Lopes, Júlio Castro; Van-Deste, Isaac; Lopes, Rui Pedro

http://hdl.handle.net/10198/29974

Use this identifier to reference this record.

Name:	Description:	Size:	Format:
Exploring_Human_Action_Recognition.pdf		2.04 MB	Adobe PDF	Download

Send Feedback

Authors

Lopes, Júlio Castro

Van-Deste, Isaac

Lopes, Rui Pedro

Abstract(s)

Through computer vision algorithms motion can be computed, which can be a crucial element to be integrated with serious game environments. To evaluate the efficacy of motion detection algorithms, the information of these algorithms can be used to perform Human Action Recognition (HAR). There are several algorithms to perform HAR, although skeleton approaches can be seen as the best way to isolate human motion. To extract the human skeleton representation, the work described in this paper evaluates three distinct methods: OpenPose (2D), YOLO-Pose (2D) and BlazePose (3D). The information translated by the skeleton representations is normalized by lightweight normalization algorithms (for further real-time application). To classify the video sequence and further action identification, a Long Short Term Memory network (LSTM), was used. Using the N-UCLA dataset, the highest F1 score of 0.745 was achieved using OpenPose skeleton extraction (2D), followed by the computation of the angles in each joint, demonstrating that the OpenPose skeleton representation can be the most viable solution for computing human motion in serious games.

Keywords

Human Action Recognition Skeleton extraction OpenPose YOLO-Pose BlazePose Moving Joints Descriptor

URI

http://hdl.handle.net/10198/29974

Citation

Lopes, Júlio Castro; Van-Deste, Isaac; Lopes, Rui Pedro (2024). Exploring Human Action Recognition for Rehabilitation Game Application. In 2024 IEEE 12th International Conference on Serious Games and Applications for Health (SeGAH). IEEE, p. 1-8. ISBN 979-8-3503-8438-3