Papers Motion Captioning
“Motion Captioning” 태그가 달린 논문 22편 · 필터 해제
Identity-Aware Human-Object Interaction Motion Captioning
Existing human-object interaction (HOI) motion captioning methods typically describe what happens while referring to the subject using generic terms such as "a person" or "someone", without grounding the caption in subje…
Motion CaptioningMotionAtlas: Detailed Region Captioning for Motion-Centric Videos
We propose MotionAtlas, a system for detailed captioning of motion-centric videos, comprising (1) a dedicated human-annotated benchmark, (2) a scalable, high-quality pipeline to construct training samples, and (3) a fami…
Motion CaptioningFine-grained Human Motion Understanding with Language Models
In this work, we propose \methodname, an LLM-based model for fine-grained human motion understanding that represents motion as a sequence of skeletal poses with explicit timestamps for each pose. Each pose encodes body j…
Question AnsweringMotion CaptioningAnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild
As wearable and mobile devices become increasingly embedded in daily life, they offer a practical way to continuously sense human motion in the wild. But inertial signals are highly dependent on the sensing setup, includ…
Cross-Modal RetrievalActivity RecognitionMotion CaptioningEncoder-Free Human Motion Understanding via Structured Motion Descriptions
The world knowledge and reasoning capabilities of text-based large language models (LLMs) are advancing rapidly, yet current approaches to human motion understanding, including motion question answering and captioning, h…
Question AnsweringMotion CaptioningUniversal Skeleton Understanding via Differentiable Rendering and MLLMs
Multimodal large language models (MLLMs) exhibit strong visual-language reasoning, yet cannot process structured, non-visual data such as human skeletons. Existing methods either compress skeleton dynamics into lossy fea…
Action RecognitionQuestion AnsweringMotion CaptioningFingerCap: Fine-grained Finger-level Hand Motion Captioning
Understanding fine-grained human hand motion is fundamental to visual perception, embodied intelligence, and multimodal communication. In this work, we propose Fine-grained Finger-level Hand Motion Captioning (FingerCap)…
Motion CaptioningDense Motion Captioning
Recent advances in 3D human motion and language integration have primarily focused on text-to-motion generation, leaving the task of motion understanding relatively unexplored. We introduce Dense Motion Captioning, a nov…
Motion CaptioningMoGIC: Boosting Motion Generation via Intention Understanding and Visual Context
Existing text-driven motion generation methods often treat synthesis as a bidirectional mapping between language and motion, but remain limited in capturing the causal logic of action execution and the human intentions t…
Motion CaptioningMotion SynthesisVisual GroundingMonSTeR: a Unified Model for Motion, Scene, Text Retrieval
Intention drives human movement in complex environments, but such movement can only happen if the surrounding context supports it. Despite the intuitive nature of this mechanism, existing research has not yet provided to…
Motion CaptioningText RetrievalHierarchical Motion Captioning Utilizing External Text Data Source
This paper introduces a novel approach to enhance existing motion captioning methods, which directly map representations of movement to high-level descriptive captions (e.g., ``a person doing jumping jacks"). The existin…
Motion CaptioningGigaHands: A Massive Annotated Dataset of Bimanual Hand Activities
Understanding bimanual human hand activities is a critical problem in AI and robotics. We cannot build large models of bimanual activities because existing datasets lack the scale, coverage of diverse hand activities, an…
DiversityMotion Captioningtext annotationMoTe: Learning Motion-Text Diffusion Model for Multiple Generation Tasks
Recently, human motion analysis has experienced great improvement due to inspiring generative models such as the denoising diffusion model and large language model. While the existing approaches mainly focus on generatin…
DecoderDenoisingLanguage ModelingLanguage Modelling+3MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understanding
Generating lifelike human motions from descriptive texts has experienced remarkable research focus in the recent years, propelled by the emerging requirements of digital humans.Despite impressive advances, existing appro…
DescriptiveLanguage ModelingLanguage ModellingMotion Captioning+2LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioning
Language plays a vital role in the realm of human motion. Existing methods have largely depended on CLIP text embeddings for motion generation, yet they fall short in effectively aligning language and motion due to CLIP'…
Large Language ModelMotion CaptioningMotion GenerationRepresentation Learning+2Transformer with Controlled Attention for Synchronous Motion Captioning
In this paper, we address a challenging task, synchronous motion captioning, that aim to generate a language description synchronized with human motion sequences. This task pertains to numerous applications, such as alig…
Action LocalizationAction SegmentationMotion CaptioningPhrase Grounding+4Motion-Agent: A Conversational Framework for Human Motion Generation with LLMs
While previous approaches to 3D human motion generation have achieved notable success, they often rely on extensive training and are limited to specific tasks. To address these challenges, we introduce Motion-Agent, an e…
Language ModelingLanguage ModellingMotion CaptioningMotion GenerationDigital Life Project: Autonomous 3D Characters with Social Intelligence
In this work, we present Digital Life Project, a framework utilizing language as the universal medium to build autonomous 3D characters, who are capable of engaging in social interactions and expressing with articulated …
DiversityMotion CaptioningMotion GenerationMotion SynthesisMotion2Language, unsupervised learning of synchronized semantic motion segmentation
In this paper, we investigate building a sequence to sequence architecture for motion to language translation and synchronization. The aim is to translate motion capture inputs into English natural-language descriptions,…
Motion CaptioningMotion SegmentationSemantic SegmentationText GenerationGuided Attention for Interpretable Motion Captioning
Diverse and extensive work has recently been conducted on text-conditioned human motion generation. However, progress in the reverse direction, motion captioning, has seen less comparable advancement. In this paper, we i…
Action LocalizationMotion CaptioningMotion GenerationSpatio-Temporal Video Grounding+1