Weak Reward Model Transforms Generative Models into Robust Causal Event Extraction Systems
The inherent ambiguity of cause and effect boundaries poses a challenge in evaluating causal event extraction tasks. Traditional metrics like Exact Match and BertScore poorly reflect model performance, so we trained evaluation models to approximate human evaluation, achieving high agreement. We used them to perform Reinforcement Learning with extraction models to align them with human preference, prioritising semantic understanding. We successfully explored our approach through multiple datasets, including transferring an evaluator trained on one dataset to another as a way to decrease the reliance on human-annotated data. In that vein, we also propose a weak-to-strong supervision method that uses a fraction of the annotated data to train an evaluation model while still achieving high performance in training an RL model. Our code is available at https://github.com/oyarsa/event_extraction/tree/causal-event-extraction.
Code (1)
Tasks
Event Causality IdentificationEvent ExtractionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Interpretable Reward Redistribution in Reinforcement Learning: A Causal Approach
A major challenge in reinforcement learning is to determine which state-action pairs are responsible for future rewards that are delayed. Reward redistribution serves as a solution to re-assign credits for each time step…
reinforcement-learningReinforcement LearningCausalDrive: Real-time Causal World Models for Autonomous Driving
World models have emerged as a promising paradigm for scaling autonomous driving (AD) data, yet existing video generative models fall short as interactive simulators. Layout-conditioned renderers rely on "oracle" future …
Reinforcement LearningAutonomous DrivingCausal KL: Evaluating Causal Discovery
The two most commonly used criteria for assessing causal model discovery with artificial data are edit-distance and Kullback-Leibler divergence, measured from the true model to the learned model. Both of these metrics ma…
Causal DiscoveryModel DiscoveryExploring Generative Process Reward Modeling for Semi-Structured Data: A Case Study of Table Question Answering
Process reward models (PRMs) enhance complex reasoning in large language models (LLMs) by evaluating candidate solutions step-by-step and selecting answers based on aggregated step scores. While effective in domains such…
Question AnsweringCorrect Looks Better: Pairwise Comparisons Reveal Accuracy Rankings
Pairwise comparisons combined with aggregation methods like Elo have become central to evaluating generative models, yet concerns remain that they reward superficial stylistic cues or display judge biases. In a more posi…