paper-with-me

Papers

Explainable Behavior Cloning: Teaching Large Language Model Agents through Learning by Demonstration

2024-10-30 · Yanchu Guan, Dong Wang, Yan Wang, Haiqing Wang, Renen Sun, Chenyi Zhuang, Jinjie Gu, Zhixuan Chu

Autonomous mobile app interaction has become increasingly important with growing complexity of mobile applications. Developing intelligent agents that can effectively navigate and interact with mobile apps remains a significant challenge. In this paper, we propose an Explainable Behavior Cloning LLM Agent (EBC-LLMAgent), a novel approach that combines large language models (LLMs) with behavior cloning by learning demonstrations to create intelligent and explainable agents for autonomous mobile app interaction. EBC-LLMAgent consists of three core modules: Demonstration Encoding, Code Generation, and UI Mapping, which work synergistically to capture user demonstrations, generate executable codes, and establish accurate correspondence between code and UI elements. We introduce the Behavior Cloning Chain Fusion technique to enhance the generalization capabilities of the agent. Extensive experiments on five popular mobile applications from diverse domains demonstrate the superior performance of EBC-LLMAgent, achieving high success rates in task completion, efficient generalization to unseen scenarios, and the generation of meaningful explanations.

📄 PDF Abstract BibTeX arXiv:2410.22916

Code (0)

등록된 구현이 없습니다.

Tasks

Code GenerationLanguage ModelingLanguage ModellingLarge Language ModelNavigate

Similar Papers 제목 키워드 기반

Optimally Teaching a Linear Behavior Cloning Agent

2023-11-26 · Shubham Kumar Bharti, Stephen Wright, Adish Singla, Xiaojin Zhu

We study optimal teaching of Linear Behavior Cloning (LBC) learners. In this setup, the teacher can select which states to demonstrate to an LBC learner. The learner maintains a version space of infinite linear hypothese…

Diverse Imitation Learning via Self-OrganizingGenerative Models

2021-09-29 · Arash Vahabpour, QIUJING LU, Tianyi Wang, Omead Pooladzandi 외

Imitation learning is the problem of teaching an agent to replicate expert policy from demonstrations when the underlying reward function is unavailable. This task becomes particularly challenging when the expert demonst…

Imitation LearningMuJoCo

Augmenting Flight Training with AI to Efficiently Train Pilots

2022-10-13 · Michael Guevarra, Srijita Das, Christabel Wayllace, Carrie Demmans Epp 외

We propose an AI-based pilot trainer to help students learn how to fly aircraft. First, an AI agent uses behavioral cloning to learn flying maneuvers from qualified flight instructors. Later, the system uses the agent's …

AI Agent

Learning Nonverbal Cues in Multiparty Social Interactions for Robotic Facilitators

2025-01-18 · Antonio Lech Martin-Ozimek, Isuru Jayarathne, Su Larb Mon, Jouhyeong Chew

Conventional behavior cloning (BC) models often struggle to replicate the subtleties of human actions. Previous studies have attempted to address this issue through the development of a new BC technique: Implicit Behavio…

Exploring the Limitations of Behavior Cloning for Autonomous Driving

2019-04-18 · ICCV 2019 10 · Felipe Codevilla, Eder Santana, Antonio M. López, Adrien Gaidon

Driving requires reacting to a wide variety of complex environment conditions and agent behaviors. Explicitly modeling each possible scenario is unrealistic. In contrast, imitation learning can, in theory, leverage data …

Autonomous DrivingImitation Learning