Towards Activated Muscle Group Estimation in the Wild
In this paper, we tackle the new task of video-based Activated Muscle Group Estimation (AMGE) aiming at identifying active muscle regions during physical activity in the wild. To this intent, we provide the MuscleMap dataset featuring >15K video clips with 135 different activities and 20 labeled muscle groups. This dataset opens the vistas to multiple video-based applications in sports and rehabilitation medicine under flexible environment constraints. The proposed MuscleMap dataset is constructed with YouTube videos, specifically targeting High-Intensity Interval Training (HIIT) physical exercise in the wild. To make the AMGE model applicable in real-life situations, it is crucial to ensure that the model can generalize well to numerous types of physical activities not present during training and involving new combinations of activated muscles. To achieve this, our benchmark also covers an evaluation setting where the model is exposed to activity types excluded from the training set. Our experiments reveal that the generalizability of existing architectures adapted for the AMGE task remains a challenge. Therefore, we also propose a new approach, TransM3E, which employs a multi-modality feature fusion mechanism between both the video transformer model and the skeleton-based graph convolution model with novel cross-modal knowledge distillation executed on multi-classification tokens. The proposed method surpasses all popular video classification models when dealing with both, previously seen and new types of physical activities. The database and code can be found at https://github.com/KPeng9510/MuscleMap.
Code (1)
Tasks
Activity RecognitionHuman Activity RecognitionKnowledge DistillationVideo ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Video-based Exercise Classification and Activated Muscle Group Prediction with Hybrid X3D-SlowFast Network
This paper introduces a simple yet effective strategy for exercise classification and muscle group activation prediction (MGAP). These tasks have significant implications for personal fitness, facilitating more affordabl…
Activity RecognitionInsomnia impairs muscle function via regulating protein degradation and muscle clock
Background: Insomnia makes people more physically unable of doing daily duties, which results in a lack of strength, leads to lacking in strength. However, the effects of insomnia on muscle function have not yet been tho…
Investigating the Correlation between Force Output, Strains, and Pressure for Active Skeletal Muscle Contractions
Experimental observations suggest that the force output of the skeletal muscle tissue can be correlated to the intra-muscular pressure generated by the muscle belly. However, pressure often proves difficult to measure th…
A deep learning pipeline for identification of motor units in musculoskeletal ultrasound
Ultrasound imaging provides information from a large part of the muscle. It has recently been shown that ultrafast ultrasound imaging can be used to record and analyze the mechanical response of individual MUs using blin…
blind source separationSingle Slice Thigh CT Muscle Group Segmentation with Domain Adaptation and Self-Training
Objective: Thigh muscle group segmentation is important for assessment of muscle anatomy, metabolic disease and aging. Many efforts have been put into quantifying muscle tissues with magnetic resonance (MR) imaging inclu…
AnatomyComputed Tomography (CT)Domain AdaptationUnsupervised Domain Adaptation