Offline Imitation Learning Through Graph Search and Retrieval

2024-07-22 · Zhao-Heng Yin, Pieter Abbeel

Imitation learning is a powerful machine learning algorithm for a robot to acquire manipulation skills. Nevertheless, many real-world manipulation tasks involve precise and dexterous robot-object interactions, which make it difficult for humans to collect high-quality expert demonstrations. As a result, a robot has to learn skills from suboptimal demonstrations and unstructured interactions, which remains a key challenge. Existing works typically use offline deep reinforcement learning (RL) to solve this challenge, but in practice these algorithms are unstable and fragile due to the deadly triad issue. To overcome this problem, we propose GSR, a simple yet effective algorithm that learns from suboptimal demonstrations through Graph Search and Retrieval. We first use pretrained representation to organize the interaction experience into a graph and perform a graph search to calculate the values of different behaviors. Then, we apply a retrieval-based procedure to identify the best behavior (actions) on each state and use behavior cloning to learn that behavior. We evaluate our method in both simulation and real-world robotic manipulation tasks with complex visual inputs, covering various precise and dexterous manipulation skills with objects of different physical properties. GSR can achieve a 10% to 30% higher success rate and over 30% higher proficiency compared to baselines. Our project page is at https://zhaohengyin.github.io/gsr.

📄 PDF Abstract BibTeX arXiv:2407.15403

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningImitation LearningReinforcement Learning (RL)Retrieval

Do We Still Need GraphRAG? Benchmarking RAG and GraphRAG for Agentic Search Systems

2026-04-01 · Dongzhe Fan, Zheyi Xue, Siyuan Liu, Qiaoyu Tan arxiv

Retrieval-augmented generation (RAG) and its graph-based extensions (GraphRAG) are effective paradigms for improving large language model (LLM) reasoning by grounding generation in external knowledge. However, most exist…

Question Answering

EvoGraph-R1: Self-Evolving Multimodal Knowledge Hypergraphs for Agentic Retrieval

2026-07-14 · Jiashi Lin, Changhong Jiang, Xiangru Lin, Ruifei Zhang 외 arxiv

Retrieval-augmented generation (RAG) has emerged as a critical paradigm for grounding Multimodal Large Language Models (MLLMs) in external knowledge. Recent GraphRAG methods introduce structured entity-relation graphs to…

Knowledge Graphs

Fast Spectral Ranking for Similarity Search

2017-03-20 · CVPR 2018 6 · Ahmet Iscen, Yannis Avrithis, Giorgos Tolias, Teddy Furon 외

Despite the success of deep learning on representing images for particular object retrieval, recent studies show that the learned representations still lie on manifolds in a high dimensional space. This makes the Euclide…

Retrieval

FastInsight: Fast and Insightful Retrieval via Fusion Operators for Graph RAG

2026-01-26 · Seonho An, Chaejeong Hyun, Min-Soo Kim arxiv

Existing Graph RAG methods aiming for insightful retrieval on corpus graphs typically rely on time-intensive processes that interleave Large Language Model (LLM) reasoning. To enable time-efficient insightful retrieval, …

GraphSearch: An Agentic Deep Searching Workflow for Graph Retrieval-Augmented Generation

2025-09-26 · Cehao Yang, Xiaojun Wu, Xueyuan Lin, Chengjin Xu 외 arxiv

Graph Retrieval-Augmented Generation (GraphRAG) enhances factual reasoning in LLMs by structurally modeling knowledge through graph-based representations. However, existing GraphRAG approaches face two core limitations: …