Reinforcement Learning in hyperbolic space for multi-step reasoning

2025-07-21 · Tao Xu, Dung-Yang Lee, Momiao Xiong arxiv

Multi-step reasoning is a fundamental challenge in artificial intelligence, with applications ranging from mathematical problem-solving to decision-making in dynamic environments. Reinforcement Learning (RL) has shown promise in enabling agents to perform multi-step reasoning by optimizing long-term rewards. However, conventional RL methods struggle with complex reasoning tasks due to issues such as credit assignment, high-dimensional state representations, and stability concerns. Recent advancements in Transformer architectures and hyperbolic geometry have provided novel solutions to these challenges. This paper introduces a new framework that integrates hyperbolic Transformers into RL for multi-step reasoning. The proposed approach leverages hyperbolic embeddings to model hierarchical structures effectively. We present theoretical insights, algorithmic details, and experimental results that include Frontier Math and nonlinear optimal control problems. Compared to RL with vanilla transformer, the hyperbolic RL largely improves accuracy by (32%~44%) on FrontierMath benchmark, (43%~45%) on nonlinear optimal control benchmark, while achieving impressive reduction in computational time by (16%~32%) on FrontierMath benchmark, (16%~17%) on nonlinear optimal control benchmark. Our work demonstrates the potential of hyperbolic Transformers in reinforcement learning, particularly for multi-step reasoning tasks that involve hierarchical structures.

📄 PDF Abstract BibTeX arXiv:2507.16864

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

GeoWorld: Geometric World Models

2026-02-26 · Zeyu Zhang, Danning Li, Ian Reid, Richard Hartley arxiv

Energy-based predictive world models provide a powerful approach for multi-step visual planning by reasoning over latent energy landscapes rather than generating pixels. However, existing approaches face two major challe…

Reinforcement Learning

HyperGuide: Hyperbolic Guidance for Efficient Multi-Step Reasoning in Large Language Models

2026-05-22 · Yuyu Liu, Haotian Xu, Yanan He, Sarang Rajendra Patil 외 arxiv

Multi-step reasoning remains a central challenge for large language models: single-pass generation is efficient but lacks accuracy; tree-search methods explore multiple paths but are computation-heavy. We address this ga…

Multi-Hop Reasoning for Question Answering with Hyperbolic Representations

2025-07-04 · Simon Welz, Lucie Flek, Akbar Karimi arxiv

Hyperbolic representations are effective in modeling knowledge graph data which is prevalently used to facilitate multi-hop reasoning. However, a rigorous and detailed comparison of the two spaces for this task is lackin…

Question Answering

Hyperbolic Deep Reinforcement Learning

2022-10-04 · Edoardo Cetin, Benjamin Chamberlain, Michael Bronstein, Jonathan J Hunt

We propose a new class of deep reinforcement learning (RL) algorithms that model latent representations in hyperbolic space. Sequential decision-making requires reasoning about the possible future consequences of current…

Decision Making Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+2

Neural-Symbolic Logic Query Answering in Non-Euclidean Space

2026-02-25 · Lihui Liu arxiv

Answering complex first-order logic (FOL) queries on knowledge graphs is essential for reasoning. Symbolic methods offer interpretability but struggle with incomplete graphs, while neural approaches generalize better but…

Knowledge Graph Completion Knowledge Graphs