Choice-75: A Dataset on Decision Branching in Script Learning
Script learning studies how stereotypical events unfold, enabling machines to reason about narratives with implicit information. Previous works mostly consider a script as a linear sequence of events while ignoring the potential branches that arise due to people's circumstantial choices. We hence propose Choice-75, the first benchmark that challenges intelligent systems to make decisions given descriptive scenarios, containing 75 scripts and more than 600 scenarios. We also present preliminary results with current large language models (LLM). Although they demonstrate overall decent performance, there is still notable headroom in hard scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
DescriptiveSimilar Papers 제목 키워드 기반
Detecting and Understanding Branching Frequency Changes in Process Models
Business processes are continuously evolving in order to adapt to changes due to various factors. One type of process changes are branching frequency changes, which are related to changes in frequencies between different…
Rushes: A Human Preference Dataset for Pluralistic Alignment
We introduce Rushes, a dataset and benchmark for studying revealed human engagement preferences in interactive narrative environments. Rushes is collected through a game interface where users interact with AI-generated b…
The Paradox of Choice: Using Attention in Hierarchical Reinforcement Learning
Decision-making AI agents are often faced with two important challenges: the depth of the planning horizon, and the branching factor due to having many choices. Hierarchical reinforcement learning methods aim to solve th…
Decision MakingHierarchical Reinforcement Learningreinforcement-learningReinforcement Learning+1Lookback for Learning to Branch
The expressive and computationally inexpensive bipartite Graph Neural Networks (GNN) have been shown to be an important component of deep learning based Mixed-Integer Linear Program (MILP) solvers. Recent works have demo…
Model SelectionVariable SelectionBranching Strategies Based on Subgraph GNNs: A Study on Theoretical Promise versus Practical Reality
Graph Neural Networks (GNNs) have emerged as a promising approach for ``learning to branch'' in Mixed-Integer Linear Programming (MILP). While standard Message-Passing GNNs (MPNNs) are efficient, they theoretically lack …