How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines

2026-04-23 · Abel Yagubyan arxiv

Large language model (LLM) agents with tool-calling capabilities are increasingly deployed in production systems, yet a fundamental reliability question remains under-explored: does the same agent behave the same way twice? We present a systematic empirical study of behavioral consistency in multi-step tool-calling agents, measuring whether agents select the same tools, in the same order, with the same arguments, across repeated identical invocations. Unlike prior work on consistency in ReAct-style agents(search-only, free-text actions), we study the richer setting of structured tool-calling interfaces with typed parameters and consequential side effects.

📄 PDF Abstract BibTeX arXiv:2605.28840

Code (0)

등록된 구현이 없습니다.

When Agents Disagree With Themselves: Measuring Behavioral Consistency in LLM-Based Agents

2026-02-12 · Aman Mehta arxiv

Run the same LLM agent on the same task twice: do you get the same behavior? We find the answer is often no. In a study of 3,000 agent runs across three models (Llama 3.1 70B, GPT-4o, and Claude Sonnet 4.5) on HotpotQA, …

Do Large Language Models Walk Their Talk? Measuring the Gap Between Implicit Associations, Self-Report, and Behavioral Altruism

2025-12-01 · Sandro Andric arxiv

We investigate whether Large Language Models (LLMs) exhibit altruistic tendencies, and critically, whether their implicit associations and self-reports predict actual altruistic behavior. Using a multi-method approach in…

Belief Without Behavior: Measuring the Translation of Theory of Mind into Coordinated Social Action in Vision-Language Models

2026-08-21 · Tonglin Yan, Gregoire Sergeant-Perthuis, David Rudrauf arxiv

Effective social interaction requires agents to translate mental state inferences into coordinated behavioral signals across verbal and nonverbal channels simultaneously. Yet existing benchmarks evaluate theory of mind (…

Measuring Policy Distance for Multi-Agent Reinforcement Learning

2024-01-20 · Tianyi Hu, Zhiqiang Pu, Xiaolin Ai, Tenghai Qiu 외

Diversity plays a crucial role in improving the performance of multi-agent reinforcement learning (MARL). Currently, many diversity-based methods have been developed to overcome the drawbacks of excessive parameter shari…

Diversity Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Measuring What Persists: Conditioning Mechanisms and a Geometric Framework for AI Agent Identity

2026-06-20 · Andrew Tanner arxiv

AI agents in long-context applications drift from their specified identity. Current methods detect this only after qualitative degradation is visible. We present a geometric framework for measuring identity structure usi…