paper-with-me

홈 › Papers

Active Reasoning in an Open-World Environment

2023-11-03 · NeurIPS 2023 11

Recent advances in vision-language learning have achieved notable success on complete-information question-answering datasets through the integration of extensive world knowledge. Yet, most models operate passively, responding to questions based on pre-stored knowledge. In stark contrast, humans possess the ability to actively explore, accumulate, and reason using both newfound and existing information to tackle incomplete-information questions. In response to this gap, we introduce $Conan$, an interactive open-world environment devised for the assessment of active reasoning. $Conan$ facilitates active exploration and promotes multi-round abductive inference, reminiscent of rich, open-world settings like Minecraft. Diverging from previous works that lean primarily on single-round deduction via instruction following, $Conan$ compels agents to actively interact with their surroundings, amalgamating new evidence with prior knowledge to elucidate events from incomplete observations. Our analysis on $Conan$ underscores the shortcomings of contemporary state-of-the-art models in active exploration and understanding complex scenarios. Additionally, we explore Abduction from Deduction, where agents harness Bayesian rules to recast the challenge of abduction as a deductive process. Through $Conan$, we aim to galvanize advancements in active reasoning and set the stage for the next generation of artificial intelligence agents adept at dynamically engaging in environments.

📄 PDF Abstract BibTeX arXiv:2311.02018

Code (0)

등록된 구현이 없습니다.

Tasks

Instruction FollowingMinecraftQuestion AnsweringWorld Knowledge

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Similar Papers 제목 키워드 기반

Relational Semantic Reasoning on 3D Scene Graphs for Open World Interactive Object Search

2026-03-05 · Imen Mahdi, Matteo Cassinelli, Fabien Despinoy, Tim Welschehold 외 arxiv

Open-world interactive object search in household environments requires understanding semantic relationships between objects and their surrounding context to guide exploration efficiently. Prior methods either rely on vi…

PhysVLM-AVR: Active Visual Reasoning for Multimodal Large Language Models in Physical Environments

2025-10-24 · Weijie Zhou, Xuantang Xiong, Yi Peng, Manli Tao 외 arxiv

Visual reasoning in multimodal large language models (MLLMs) has primarily been studied in static, fully observable settings, limiting their effectiveness in real-world environments where information is often incomplete …

Visual Reasoning

Counterfactual Reasoning and Environment Design for Active Preference Learning

2025-07-07 · Yi-Shiuan Tung, Bradley Hayes, Alessandro Roncone arxiv

For effective real-world deployment, robots should adapt to human preferences, such as balancing distance, time, and safety in delivery routing. Active preference learning (APL) learns human reward functions by presentin…

Affordance-Aware Interactive Decision-Making and Execution for Ambiguous Instructions

2026-02-05 · Hengxuan Xu, Fengbo Lan, Zhixin Zhao, Shengjie Wang 외 arxiv

Enabling robots to explore and act in unfamiliar environments under ambiguous human instructions by interactively identifying task-relevant objects (e.g., identifying cups or beverages for "I'm thirsty") remains challeng…

Towards Open Environments and Instructions: General Vision-Language Navigation via Fast-Slow Interactive Reasoning

2026-01-14 · Yang Li, Aming Wu, Zihao Zhang, Yahong Han arxiv

Vision-Language Navigation (VLN) aims to enable agents to navigate to a target location based on language instructions. Traditional VLN often follows a close-set assumption, i.e., training and test data share the same st…

Vision-Language Navigation