Muscles in Action

2022-12-05 · ICCV 2023 1 · Mia Chiquier, Carl Vondrick

Human motion is created by, and constrained by, our muscles. We take a first step at building computer vision methods that represent the internal muscle activity that causes motion. We present a new dataset, Muscles in Action (MIA), to learn to incorporate muscle activity into human motion representations. The dataset consists of 12.5 hours of synchronized video and surface electromyography (sEMG) data of 10 subjects performing various exercises. Using this dataset, we learn a bidirectional representation that predicts muscle activation from video, and conversely, reconstructs motion from muscle activation. We evaluate our model on in-distribution subjects and exercises, as well as on out-of-distribution subjects and exercises. We demonstrate how advances in modeling both modalities jointly can serve as conditioning for muscularly consistent motion generation. Putting muscles into computer vision systems will enable richer models of virtual humans, with applications in sports, fitness, and AR/VR.

📄 PDF Abstract BibTeX arXiv:2212.02978

Code (0)

등록된 구현이 없습니다.

Tasks

Motion Generation

I am Robot: Neuromuscular Reinforcement Learning to Actuate Human Limbs through Functional Electrical Stimulation

2021-03-09 · Nat Wannawas, Ali Shafti, A. Aldo Faisal

Human movement disorders or paralysis lead to the loss of control of muscle activation and thus motor control. Functional Electrical Stimulation (FES) is an established and safe technique for contracting muscles by stimu…

Deep Reinforcement Learning Reinforcement Learning (RL)

Stuttering Speech Disfluency Prediction using Explainable Attribution Vectors of Facial Muscle Movements

2020-10-02 · Arun Das, Jeffrey Mock, Henry Chacon, Farzan Irani 외

Speech disorders such as stuttering disrupt the normal fluency of speech by involuntary repetitions, prolongations and blocking of sounds and syllables. In addition to these disruptions to speech fluency, most adults who…

Blocking Explainable Artificial Intelligence (XAI)

Unsupervised Anomaly Detection on Implicit Shape representations for Sarcopenia Detection

2025-02-13 · Louise Piecuch, Jeremie Huet, Antoine Frouin, Antoine Nordez 외

Sarcopenia is an age-related progressive loss of muscle mass and strength that significantly impacts daily life. A commonly studied criterion for characterizing the muscle mass has been the combination of 3D imaging and …

Anomaly Detection Unsupervised Anomaly Detection

Automated Segmentation of Hip and Thigh Muscles in Metal Artifact-Contaminated CT using Convolutional Neural Network-Enhanced Normalized Metal Artifact Reduction

2019-06-27 · Mitsuki Sakamoto, Yuta Hiasa, Yoshito Otake, Masaki Takao 외

In total hip arthroplasty, analysis of postoperative medical images is important to evaluate surgical outcome. Since Computed Tomography (CT) is most prevalent modality in orthopedic surgery, we aimed at the analysis of …

Computed Tomography (CT)Metal Artifact Reduction Segmentation

Detecting muscle activation using ultrasound speed of sound inversion with deep learning

2019-10-20 · Micha Feigin, Manuel Zwecker, Daniel Freedman, Brian W. Anthony

Functional muscle imaging is essential for diagnostics of a multitude of musculoskeletal afflictions such as degenerative muscle diseases, muscle injuries, muscle atrophy, and neurological related issues such as spastici…