Hallucinated Humans as the Hidden Context for Labeling 3D Scenes
For scene understanding, one popular approach has been to model the object-object relationships. In this paper, we hypothesize that such relationships are only an artifact of certain hidden factors, such as humans. For example, the objects, monitor and keyboard, are strongly spatially correlated only because a human types on the keyboard while watching the monitor. Our goal is to learn this hidden human context (i.e., the human-object relationships), and also use it as a cue for labeling the scenes. We present Infinite Factored Topic Model (IFTM), where we consider a scene as being generated from two types of topics: human configurations and human-object relationships. This enables our algorithm to hallucinate the possible configurations of the humans in the scene parsimoniously. Given only a dataset of scenes containing objects but not humans, we show that our algorithm can recover the human object relationships. We then test our algorithm on the task of attribute and object labeling in 3D scenes and show consistent improvements over the state-of-the-art.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeObjectScene UnderstandingSimilar Papers 제목 키워드 기반
Understanding Human Context in 3D Scenes by Learning Spatial Affordances with Virtual Skeleton Models
Robots are often required to operate in environments where humans are not present, but yet require the human context information for better human-robot interaction. Even when humans are present in the environment, detect…
Multi-Label ClassificationMUlTI-LABEL-ClASSIFICATIONHow do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study
While AI-generated hallucinations pose considerable risks, the underlying cognitive mechanisms by which humans can successfully recognize or be misled by these hallucinations remain unclear. To address this problem, this…
Fact VerificationWho Brings the Frisbee: Probing Hidden Hallucination Factors in Large Vision-Language Model via Causality Analysis
Recent advancements in large vision-language models (LVLM) have significantly enhanced their ability to comprehend visual inputs alongside natural language. However, a major challenge in their real-world application is h…
HallucinationLanguage ModelingLanguage ModellingSemantic Labeling of 3D Point Clouds for Indoor Scenes
Inexpensive RGB-D cameras that give an RGB image together with depth data have become widely available. In this paper, we use this data to build 3D point clouds of full indoor scenes such as an office and address the tas…
ObjectUnderstanding the Role of Individual Units in a Deep Neural Network
Deep neural networks excel at finding hierarchical representations that solve complex tasks over large data sets. How can we humans understand these learned representations? In this work, we present network dissection, a…
Generative Adversarial Networkimage-classificationImage ClassificationImage Generation+1