Goals and the Structure of Experience

2025-08-20 · Nadav Amir, Stas Tiomkin, Angela Langdon arxiv

Purposeful behavior is a hallmark of natural and artificial intelligence. Its acquisition is often believed to rely on world models, comprising both descriptive (what is) and prescriptive (what is desirable) aspects that identify and evaluate state of affairs in the world, respectively. Canonical computational accounts of purposeful behavior, such as reinforcement learning, posit distinct components of a world model comprising a state representation (descriptive aspect) and a reward function (prescriptive aspect). However, an alternative possibility, which has not yet been computationally formulated, is that these two aspects instead co-emerge interdependently from an agent's goal. Here, we describe a computational framework of goal-directed state representation in cognitive agents, in which the descriptive and prescriptive aspects of a world model co-emerge from agent-environment interaction sequences, or experiences. Drawing on Buddhist epistemology, we introduce a construct of goal-directed, or telic, states, defined as classes of goal-equivalent experience distributions. Telic states provide a parsimonious account of goal-directed learning in terms of the statistical divergence between behavioral policies and desirable experience features. We review empirical and theoretical literature supporting this novel perspective and discuss its potential to provide a unified account of behavioral, phenomenological and neural dimensions of purposeful behaviors across diverse substrates.

📄 PDF Abstract BibTeX arXiv:2508.15013

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Towards Optimizing OCR for Accessibility

2022-06-21 · Peya Mowar, Tanuja Ganu, Saikat Guha

Visual cues such as structure, emphasis, and icons play an important role in efficient information foraging by sighted individuals and make for a pleasurable reading experience. Blind, low-vision and other print-disabled…

Optical Character Recognition (OCR)text-to-speech Text to Speech

Tree-of-Experience: A Structured Experience-Management Solution for Self-Evolving Agents under Low-Repetition and Implicit-Reward Environments

2026-06-05 · Zihao Deng, Yining Zhu, Leiming Wang, Jingfei Lu 외 arxiv

Experience-based self-evolution is crucial for LLM agents, but existing benchmarks often assume explicit goals, stable task patterns, and clear feedback. We study a more challenging setting: low-repetition tasks with imp…

DHER: Hindsight Experience Replay for Dynamic Goals

2019-05-01 · ICLR 2019 5 · Meng Fang, Cheng Zhou, Bei Shi, Boqing Gong 외

Dealing with sparse rewards is one of the most important challenges in reinforcement learning (RL), especially when a goal is dynamic (e.g., to grasp a moving object). Hindsight experience replay (HER) has been shown an …

Object Tracking Reinforcement Learning Reinforcement Learning (RL)

MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffer

2022-06-20 · Jeewon Jeon, Woojun Kim, Whiyoung Jung, Youngchul Sung

In this paper, we consider cooperative multi-agent reinforcement learning (MARL) with sparse reward. To tackle this problem, we propose a novel method named MASER: MARL with subgoals generated from experience replay buff…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning (RL)+2

Enhancing Human Experience in Human-Agent Collaboration: A Human-Centered Modeling Approach Based on Positive Human Gain

2024-01-28 · Yiming Gao, Feiyu Liu, Liang Wang, Zhenjie Lian 외

Existing game AI research mainly focuses on enhancing agents' abilities to win games, but this does not inherently make humans have a better experience when collaborating with these agents. For example, agents may domina…

Human Agent Collaboration