paper-with-me

Papers

OPAL: Encoding Causal Understanding of Physical Systems for Robot Learning

2025-04-09 · Daniel Tcheurekdjian, Joshua Klasmeier, Tom Cooney, Christopher McCann, Tyler Fenstermaker

We present OPAL (Operant Physical Agent with Language), a novel vision-language-action architecture that introduces topological constraints to flow matching for robotic control. To do so, we further introduce topological attention. Our approach models action sequences as topologically-structured representations with non-trivial constraints. Experimental results across 10 complex manipulation tasks demonstrate OPAL's superior performance compared to previous approaches, including Octo, OpenVLA, and ${\pi}$0. Our architecture achieves significant improvements in zero-shot performance without requiring task-specific fine-tuning, while reducing inference computational requirements by 42%. The theoretical guarantees provided by our topological approach result in more coherent long-horizon action sequences. Our results highlight the potential of constraining the search space of learning problems in robotics by deriving from fundamental physical laws, and the possibility of using topological attention to embed causal understanding into transformer architectures.

📄 PDF Abstract BibTeX arXiv:2504.06538

Code (0)

등록된 구현이 없습니다.

Tasks

Vision-Language-Action

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Attention 설명 없음

Similar Papers 제목 키워드 기반

COPAL-ID: Indonesian Language Reasoning with Local Culture and Nuances

2023-11-02 · Haryo Akbarianto Wibowo, Erland Hilman Fuadi, Made Nindyatama Nityasya, Radityo Eko Prasojo 외

We present COPAL-ID, a novel, public Indonesian language common sense reasoning dataset. Unlike the previous Indonesian COPA dataset (XCOPA-ID), COPAL-ID incorporates Indonesian local and cultural nuances, and therefore,…

Common Sense Reasoning

AudioPaLM: A Large Language Model That Can Speak and Listen

2023-06-22 · Paul K. Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen, Ankur Bapna 외

We introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a un…

Language ModelingLanguage ModellingLarge Language Modelspeech-recognition+5

Computational Spectral Imaging with Unified Encoding Model: A Comparative Study and Beyond

2023-12-20 · Xinyuan Liu, Lizhi Wang, Lingen Li, Chang Chen 외

Computational spectral imaging is drawing increasing attention owing to the snapshot advantage, and amplitude, phase, and wavelength encoding systems are three types of representative implementations. Fairly comparing an…

Decoder

CausalVQA: A Physically Grounded Causal Reasoning Benchmark for Video Models

2025-06-11 · Aaron Foss, Chloe Evans, Sasha Mitts, Koustuv Sinha 외

We introduce CausalVQA, a benchmark dataset for video question answering (VQA) composed of question-answer pairs that probe models' understanding of causality in the physical world. Existing VQA benchmarks either tend to…

counterfactualDescriptiveQuestion AnsweringVideo Question Answering+1

Effort-Optimized, Accuracy-Driven Labelling and Validation of Test Inputs for DL Systems: A Mixed-Integer Linear Programming Approach

2025-07-07 · Mohammad Hossein Amini, Mehrdad Sabetzadeh, Shiva Nejati arxiv

Software systems increasingly include AI components based on deep learning (DL). Reliable testing of such systems requires near-perfect test-input validity and label accuracy, with minimal human effort. Yet, the DL commu…