ActionFlow: Equivariant, Accurate, and Efficient Policies with Spatially Symmetric Flow Matching

2024-09-06 · Niklas Funk, Julen Urain, Joao Carvalho, Vignesh Prasad, Georgia Chalvatzaki, Jan Peters

Spatial understanding is a critical aspect of most robotic tasks, particularly when generalization is important. Despite the impressive results of deep generative models in complex manipulation tasks, the absence of a representation that encodes intricate spatial relationships between observations and actions often limits spatial generalization, necessitating large amounts of demonstrations. To tackle this problem, we introduce a novel policy class, ActionFlow. ActionFlow integrates spatial symmetry inductive biases while generating expressive action sequences. On the representation level, ActionFlow introduces an SE(3) Invariant Transformer architecture, which enables informed spatial reasoning based on the relative SE(3) poses between observations and actions. For action generation, ActionFlow leverages Flow Matching, a state-of-the-art deep generative model known for generating high-quality samples with fast inference - an essential property for feedback control. In combination, ActionFlow policies exhibit strong spatial and locality biases and SE(3)-equivariant action generation. Our experiments demonstrate the effectiveness of ActionFlow and its two main components on several simulated and real-world robotic manipulation tasks and confirm that we can obtain equivariant, accurate, and efficient policies with spatially symmetric flow matching. Project website: https://flowbasedpolicies.github.io/

📄 PDF Abstract BibTeX arXiv:2409.04576

Code (0)

등록된 구현이 없습니다.

Tasks

Action GenerationSpatial Reasoning

Methods 이 논문이 사용한 방법론

Attention 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Position-Wise Feed-Forward Layer 설명 없음

EquiContact: A Hierarchical SE(3) Vision-to-Force Equivariant Policy for Spatially Generalizable Contact-rich Tasks

2025-07-15 · Joohwan Seo, Arvind Kruthiventy, Soomi Lee, Megan Teng 외 arxiv

This paper presents a framework for learning vision-based robotic policies for contact-rich manipulation tasks that generalize spatially across task configurations. We focus on achieving robust spatial generalization of …

ActionFlow: A Pipelined Action Acceleration for Vision Language Models on Edge

2025-12-23 · Yuntao Dai, Hang Gu, Teng Wang, Qianyu Cheng 외 arxiv

Vision-Language-Action (VLA) models have emerged as a unified paradigm for robotic perception and control, enabling emergent generalization and long-horizon task execution. However, their deployment in dynamic, real-worl…

Rotation Equivariant Operators for Machine Learning on Scalar and Vector Fields

2021-08-21 · Paul Shen, Michael Herbst, Venkat Viswanathan

We develop theory and software for rotation equivariant operators on scalar and vector fields, with diverse applications in simulation, optimization and machine learning. Rotation equivariance (covariance) means all fiel…

BIG-bench Machine Learning Translation

Group Equivariant Deep Reinforcement Learning

2020-07-01 · Arnab Kumar Mondal, Pratheeksha Nair, Kaleem Siddiqi

In Reinforcement Learning (RL), Convolutional Neural Networks(CNNs) have been successfully applied as function approximators in Deep Q-Learning algorithms, which seek to learn action-value functions and policies in vario…

Deep Reinforcement Learning Inductive Bias Q-Learning reinforcement-learning+2

Breaking the Symmetry: Resolving Symmetry Ambiguities in Equivariant Neural Networks

2022-10-29 · Sidhika Balachandar, Adrien Poulenard, Congyue Deng, Leonidas Guibas

Equivariant networks have been adopted in many 3-D learning areas. Here we identify a fundamental limitation of these networks: their ambiguity to symmetries. Equivariant networks cannot complete symmetry-dependent tasks…