Two Body Problem: Collaborative Visual Task Completion
Collaboration is a necessary skill to perform tasks that are beyond one agent's capabilities. Addressed extensively in both conventional and modern AI, multi-agent collaboration has often been studied in the context of simple grid worlds. We argue that there are inherently visual aspects to collaboration which should be studied in visually rich environments. A key element in collaboration is communication that can be either explicit, through messages, or implicit, through perception of the other agents and the visual world. Learning to collaborate in a visual environment entails learning (1) to perform the task, (2) when and what to communicate, and (3) how to act based on these communications and the perception of the visual world. In this paper we study the problem of learning to collaborate directly from pixels in AI2-THOR and demonstrate the benefits of explicit and implicit modes of communication to perform visual tasks. Refer to our project page for more details: https://prior.allenai.org/projects/two-body-problem
Code (0)
등록된 구현이 없습니다.
Tasks
Task 2Vocal Bursts Valence PredictionSimilar Papers 제목 키워드 기반
Completion $\neq$ Collaboration: Scaling Collaborative Effort with Agents
Current evaluations of agents remain centered around one-shot task completion, failing to account for the inherently iterative and collaborative nature of many real-world problems, where human goals are often underspecif…
Stable Diffusion-Based Approach for Human De-Occlusion
Humans can infer the missing parts of an occluded object by leveraging prior knowledge and visible cues. However, enabling deep learning models to accurately predict such occluded regions remains a challenging task. De-o…
Visual Question Answering3D Human Reconstruction2D Pose EstimationRecommendation via matrix completion using Kolmogorov complexity
A usual way to model a recommendation system is as a matrix completion problem. There are several matrix completion methods, typically using optimization approaches or collaborative filtering. Most approaches assume that…
Collaborative FilteringMatrix CompletionRecommendation SystemsNonparametric Preference Completion
We consider the task of collaborative preference completion: given a pool of items, a pool of users and a partially observed item-user rating matrix, the goal is to recover the \emph{personalized ranking} of each user ov…
Face Completion with Semantic Knowledge and Collaborative Adversarial Learning
Unlike a conventional background inpainting approach that infers a missing area from image patches similar to the background, face completion requires semantic knowledge about the target object for realistic outputs. Cur…
Facial InpaintingImage InpaintingSemantic Segmentation