Papers Action Generation
“Action Generation” 태그가 달린 논문 111편 · 필터 해제
VITA: Vision-to-Action Flow Matching Policy
We present VITA, a Vision-To-Action flow matching policy that evolves latent visual representations into latent actions for visuomotor control. Traditional flow matching and diffusion policies sample from standard source…
Action GenerationWorldVLA: Towards Autoregressive Action World Model
We present WorldVLA, an autoregressive action world model that unifies action and image understanding and generation. Our WorldVLA intergrates Vision-Language-Action (VLA) model and world model in one single framework. T…
Action GenerationmodelVision-Language-ActionFlow-Based Single-Step Completion for Efficient and Expressive Policy Learning
Generative models such as diffusion and flow-matching offer expressive policies for offline reinforcement learning (RL) by capturing rich, multimodal action distributions, but their iterative sampling introduces high inf…
Action GenerationDecision MakingOffline RLReinforcement Learning (RL)+1Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trends
Vision-language-action (VLA) models extend vision-language models (VLM) by integrating action generation modules for robotic manipulation. Leveraging strengths of VLM in vision perception and instruction understanding, V…
Action GenerationVision-Language-ActionVLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Models
Recent studies on Vision-Language-Action (VLA) models have shifted from the end-to-end action-generation paradigm toward a pipeline involving task planning followed by action generation, demonstrating improved performanc…
Action GenerationContinual LearningTask PlanningVision-Language-ActionCodeDiffuser: Attention-Enhanced Diffusion Policy via VLM-Generated Code for Instruction Ambiguity
Natural language instructions for robotic manipulation tasks often exhibit ambiguity and vagueness. For instance, the instruction "Hang a mug on the mug tree" may involve multiple valid actions if there are several mugs …
Action GenerationContact-rich ManipulationImitation LearningAutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning
Recent advancements in Vision-Language-Action (VLA) models have shown promise for end-to-end autonomous driving by leveraging world knowledge and reasoning capabilities. However, current VLA models often struggle with ph…
Action GenerationAutonomous DrivingTrajectory PlanningVision-Language-Action+1Block-wise Adaptive Caching for Accelerating Diffusion Policy
Diffusion Policy has demonstrated strong visuomotor modeling capabilities, but its high computational cost renders it impractical for real-time robotic control. Despite huge redundancy across repetitive denoising steps, …
Action GenerationDenoisingVision-Language-ActionTime-Unified Diffusion Policy with Action Discrimination for Robotic Manipulation
In many complex scenarios, robotic manipulation relies on generative models to estimate the distribution of multiple successful actions. As the diffusion model has better training robustness than other generative models,…
Action GenerationAction RecognitionDenoisingImitation LearningFreqPolicy: Efficient Flow-based Visuomotor Policy via Frequency Consistency
Generative modeling-based visuomotor policies have been widely adopted in robotic manipulation attributed to their ability to model multimodal action distributions. However, the high inference cost of multi-step sampling…
Action GenerationImage GenerationVision-Language-ActionAn Open-Source Software Toolkit & Benchmark Suite for the Evaluation and Adaptation of Multimodal Action Models
Recent innovations in multimodal action models represent a promising direction for developing general-purpose agentic systems, combining visual understanding, language comprehension, and action generation. We introduce M…
Action GenerationImage CaptioningQuestion AnsweringVision-Language-Action+1Time to Talk: LLM Agents for Asynchronous Group Communication in Mafia Games
LLMs are used predominantly in synchronous communication, where a human user and a model communicate in alternating turns. In contrast, many real-world settings are inherently asynchronous. For example, in group chats, o…
Action GenerationAsynchronous Group CommunicationGame of MafiaScheduling+1OWMM-Agent: Open World Mobile Manipulation With Multi-modal Agentic Data Synthesis
The rapid progress of navigation, manipulation, and vision models has made mobile manipulators capable in many specialized tasks. However, the open-world mobile manipulation (OWMM) task remains a challenge due to the nee…
Action GenerationDecision MakingHallucinationScene Understanding+1STAR: Learning Diverse Robot Skill Abstractions through Rotation-Augmented Vector Quantization
Transforming complex actions into discrete skill abstractions has demonstrated strong potential for robotic manipulation. Existing approaches mainly leverage latent variable models, e.g., VQ-VAE, to learn skill abstracti…
Action GenerationQuantizationSmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
Vision-language models (VLMs) pretrained on large-scale multimodal datasets encode rich visual and linguistic knowledge, making them a strong foundation for robotics. Rather than training robotic policies from scratch, r…
Action GenerationGPUVision-Language-ActionTowards a Generalizable Bimanual Foundation Policy via Flow-based Video Prediction
Learning a generalizable bimanual manipulation policy is extremely challenging for embodied agents due to the large action space and the need for coordinated arm movements. Existing approaches rely on Vision-Language-Act…
Action GenerationOptical Flow EstimationVideo PredictionVision-Language-ActionHierarchical Instruction-aware Embodied Visual Tracking
User-Centric Embodied Visual Tracking (UC-EVT) presents a novel challenge for reinforcement learning-based models due to the substantial gap between high-level user instructions and low-level agent actions. While recent …
Action GenerationPositionVisual TrackingProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos
Real-world driving requires people to observe the current environment, anticipate the future, and make appropriate driving decisions. This requirement is aligned well with the capabilities of world models, which understa…
Action GenerationAutonomous DrivingFuture predictionPrediction+1Distilling LLM Agent into Small Models with Retrieval and Code Tools
Large language models (LLMs) excel at complex reasoning tasks but remain computationally expensive, limiting their practical deployment. To address this, recent works have focused on distilling reasoning capabilities int…
Action GenerationDomain GeneralizationRetrievalFinePhys: Fine-grained Human Action Generation by Explicitly Incorporating Physical Laws for Effective Skeletal Guidance
Despite significant advances in video generation, synthesizing physically plausible human actions remains a persistent challenge, particularly in modeling fine-grained semantics and complex temporal dynamics. For instanc…
Action GenerationHuman action generationIn-Context LearningVideo Generation