Automatic Deduction Path Learning via Reinforcement Learning with Environmental Correction
Automatic bill payment is an important part of business operations in fintech companies. The practice of deduction was mainly based on the total amount or heuristic search by dividing the bill into smaller parts to deduct as much as possible. This article proposes an end-to-end approach of automatically learning the optimal deduction paths (deduction amount in order), which reduces the cost of manual path design and maximizes the amount of successful deduction. Specifically, in view of the large search space of the paths and the extreme sparsity of historical successful deduction records, we propose a deep hierarchical reinforcement learning approach which abstracts the action into a two-level hierarchical space: an upper agent that determines the number of steps of deductions each day and a lower agent that decides the amount of deduction at each step. In such a way, the action space is structured via prior knowledge and the exploration space is reduced. Moreover, the inherited information incompleteness of the business makes the environment just partially observable. To be precise, the deducted amounts indicate merely the lower bounds of the available account balance. To this end, we formulate the problem as a partially observable Markov decision problem (POMDP) and employ an environment correction algorithm based on the characteristics of the business. In the world's largest electronic payment business, we have verified the effectiveness of this scheme offline and deployed it online to serve millions of users.
Code (0)
등록된 구현이 없습니다.
Tasks
Heuristic SearchHierarchical Reinforcement Learningreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
H-GRPO: Permutation-Invariant Reinforcement Learning for Grounded Visual Reasoning
Vision-Language Models (VLMs) often achieve high performance on benchmarks while remaining "black boxes", yet they remain prone to hallucination or rely on superficial shortcuts. In this work, we propose a framework desi…
Reinforcement LearningVisual ReasoningReinforcing Dual-Path Reasoning in Spatial Vision Language Models
Spatial VLMs have made substantial progress in geometric perception, yet complex spatial reasoning requiring multi-step inference over depth, distance, and scene relations remains challenging. Moreover, different spatial…
Reinforcement LearningSpatial ReasoningBeyond 'Aha!': Toward Systematic Meta-Abilities Alignment in Large Reasoning Models
Large reasoning models (LRMs) already possess a latent capacity for long chain-of-thought reasoning. Prior work has shown that outcome-based reinforcement learning (RL) can incidentally elicit advanced reasoning behavior…
Mathreinforcement-learningReinforcement LearningReinforcement Learning (RL)Four Geometry Problems to Introduce Automated Deduction in Secondary Schools
The introduction of automated deduction systems in secondary schools face several bottlenecks, the absence of the subject of rigorous mathematical demonstrations in the curricula, the lack of knowledge by the teachers ab…
World-Model-Augmented Web Agents with Action Correction
Web agents based on large language models have demonstrated promising capability in automating web tasks. However, current web agents struggle to reason out sensible actions due to the limitations of predicting environme…