paper-with-me

홈 › Papers

From Language Models to Practical Self-Improving Computer Agents

2024-04-18 · Alex Sheng

We develop a simple and straightforward methodology to create AI computer agents that can carry out diverse computer tasks and self-improve by developing tools and augmentations to enable themselves to solve increasingly complex tasks. As large language models (LLMs) have been shown to benefit from non-parametric augmentations, a significant body of recent work has focused on developing software that augments LLMs with various capabilities. Rather than manually developing static software to augment LLMs through human engineering effort, we propose that an LLM agent can systematically generate software to augment itself. We show, through a few case studies, that a minimal querying loop with appropriate prompt engineering allows an LLM to generate and use various augmentations, freely extending its own capabilities to carry out real-world computer tasks. Starting with only terminal access, we prompt an LLM agent to augment itself with retrieval, internet search, web navigation, and text editor capabilities. The agent effectively uses these various tools to solve problems including automated software development and web-based tasks.

📄 PDF Abstract BibTeX arXiv:2404.11964

Code (0)

등록된 구현이 없습니다.

Tasks

Prompt EngineeringRetrieval

Similar Papers 제목 키워드 기반

OS-Copilot: Towards Generalist Computer Agents with Self-Improvement

2024-02-12 · Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng 외

Autonomous interaction with the computer has been a longstanding challenge with great potential, and the recent proliferation of large language models (LLMs) has markedly accelerated progress in building digital agents. …

Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agent

2025-08-06 · Yuhao Cheng, Liang Tang, Shuxian Li, Yukang Huo 외 arxiv

Computer use agents represent an emerging area in artificial intelligence, aiming to operate computers autonomously to fulfill user tasks, attracting significant attention from both industry and academia. However, the pe…

Reinforcement Learning

EvoCUA-1.5: Online Reinforcement Learning for Multi-turn Computer-Use Agents

2026-07-07 · Mianqiu Huang, Taofeng Xue, Chong Peng, Jinrui Ding 외 arxiv

Computer-use agents must solve long-horizon tasks through repeated interaction with partially observable, multimodal desktop environments. Although imitation learning and offline trajectory refinement provide strong prio…

Hybrid Self-evolving Structured Memory for GUI Agents

2026-03-11 · Sibo Zhu, Wenyi Wu, Kun Zhou, Stephen Wang 외 arxiv

The remarkable progress of vision-language models (VLMs) has enabled GUI agents to interact with computers in a human-like manner. Yet real-world computer-use tasks remain difficult due to long-horizon workflows, diverse…

SelfGoal: Your Language Agents Already Know How to Achieve High-level Goals

2024-06-07 · Ruihan Yang, Jiangjie Chen, Yikai Zhang, Siyu Yuan 외

Language agents powered by large language models (LLMs) are increasingly valuable as decision-making tools in domains such as gaming and programming. However, these agents often face challenges in achieving high-level go…

Decision Making