JECC: Commonsense Reasoning Tasks Derived from Interactive Fictions
Commonsense reasoning simulates the human ability to make presumptions about our physical world, and it is an essential cornerstone in building general AI systems. We propose a new commonsense reasoning dataset based on human's Interactive Fiction (IF) gameplay walkthroughs as human players demonstrate plentiful and diverse commonsense reasoning. The new dataset provides a natural mixture of various reasoning types and requires multi-hop reasoning. Moreover, the IF game-based construction procedure requires much less human interventions than previous ones. Different from existing benchmarks, our dataset focuses on the assessment of functional commonsense knowledge rules rather than factual knowledge. Hence, in order to achieve higher performance on our tasks, models need to effectively utilize such functional knowledge to infer the outcomes of actions, rather than relying solely on memorizing facts. Experiments show that the introduced dataset is challenging to previous machine reading models as well as the new large language models with a significant 20% performance gap compared to human experts.
Code (1)
Tasks
Reading ComprehensionSimilar Papers 제목 키워드 기반
Deriving Commonsense Inference Tasks from Interactive Fictions
Commonsense reasoning simulates the human ability to make presumptions about our physical world, and it is an indispensable cornerstone in building general AI systems. We propose a new commonsense reasoning dataset based…
Reading ComprehensionCommonsense-Focused Dialogues for Response Generation: An Empirical Study
Smooth and effective communication requires the ability to perform latent or explicit commonsense inference. Prior commonsense reasoning benchmarks (such as SocialIQA and CommonsenseQA) mainly focus on the discriminative…
Response GenerationText GenerationCommonsense Reasoning for Natural Language Processing
Commonsense knowledge, such as knowing that {``}bumping into people annoys them{''} or {``}rain makes the road slippery{''}, helps humans navigate everyday situations seamlessly. Yet, endowing machines with such human-li…
NavigateSToLa: Self-Adaptive Touch-Language Framework with Tactile Commonsense Reasoning in Open-Ended Scenarios
This paper explores the challenges of integrating tactile sensing into intelligent systems for multimodal reasoning, particularly in enabling commonsense reasoning about the open-ended physical world. We identify two key…
DiversityMixture-of-ExpertsMultimodal ReasoningFusing Pre-Trained Language Models With Multimodal Prompts Through Reinforcement Learning
Language models are capable of commonsense reasoning: while domain-specific models can learn from explicit knowledge (e.g. commonsense graphs [6], ethical norms [25]), and larger models like GPT-3 manifest broad comm…
Language ModelingLanguage Modellingreinforcement-learningReinforcement Learning+2