Transductive Reward Inference on Graph
In this study, we present a transductive inference approach on that reward information propagation graph, which enables the effective estimation of rewards for unlabelled data in offline reinforcement learning. Reward inference is the key to learning effective policies in practical scenarios, while direct environmental interactions are either too costly or unethical and the reward functions are rarely accessible, such as in healthcare and robotics. Our research focuses on developing a reward inference method based on the contextual properties of information propagation on graphs that capitalizes on a constrained number of human reward annotations to infer rewards for unlabelled data. We leverage both the available data and limited reward annotations to construct a reward propagation graph, wherein the edge weights incorporate various influential factors pertaining to the rewards. Subsequently, we employ the constructed graph for transductive reward inference, thereby estimating rewards for unlabelled data. Furthermore, we establish the existence of a fixed point during several iterations of the transductive inference process and demonstrate its at least convergence to a local optimum. Empirical evaluations on locomotion and robotic manipulation tasks validate the effectiveness of our approach. The application of our inferred rewards improves the performance in offline reinforcement learning tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Extending Transductive Knowledge Graph Embedding Models for Inductive Logical Relational Inference
Many downstream inference tasks for knowledge graphs, such as relation prediction, have been handled successfully by knowledge graph embedding techniques in the transductive setting. To address the inductive setting wher…
Graph EmbeddingGraph Neural NetworkInductive Relation PredictionKnowledge Graph Completion+3Transductive Inference and Semi-Supervised Learning
This chapter contains sections titled: Problem Settings, Problem of Generalization in Inductive and Transductive Inference, Structure of the VC Bounds and Transductive Inference, The Symmetrization Lemma and Transductive…
LEMMAGraph Transductive Defense: a Two-Stage Defense for Graph Membership Inference Attacks
Graph neural networks (GNNs) have become instrumental in diverse real-world applications, offering powerful graph learning capabilities for tasks such as social networks and medical data analysis. Despite their successes…
Graph LearningInductive LearningTransductive LearningLearning to Extrapolate Knowledge: Transductive Few-shot Out-of-Graph Link Prediction
Many practical graph problems, such as knowledge graph construction and drug-drug interaction prediction, require to handle multi-relational graphs. However, handling real-world multi-relational graphs with Graph Neural …
graph constructionKnowledge Graph CompletionLink PredictionMeta-Learning+1Local Propagation for Few-Shot Learning
The challenge in few-shot learning is that available data is not enough to capture the underlying distribution. To mitigate this, two emerging directions are (a) using local image representations, essentially multiplying…
Few-Shot Learning