Compositional Zero-Shot Learning for Attribute-Based Object Reference in Human-Robot Interaction
Language-enabled robots have been widely studied over the past years to enable natural human-robot interaction and teaming in various real-world applications. Language-enabled robots must be able to comprehend referring expressions to identify a particular object from visual perception using a set of referring attributes extracted from natural language. However, visual observations of an object may not be available when it is referred to, and the number of objects and attributes may also be unbounded in open worlds. To address the challenges, we implement an attribute-based compositional zero-shot learning method that uses a list of attributes to perform referring expression comprehension in open worlds. We evaluate the approach on two datasets including the MIT-States and the Clothing 16K. The preliminary experimental results show that our implemented approach allows a robot to correctly identify the objects referred to by human commands.
Code (0)
등록된 구현이 없습니다.
Tasks
16kAttributeCompositional Zero-Shot LearningReferring ExpressionReferring Expression ComprehensionZero-Shot LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning to Compose Soft Prompts for Compositional Zero-Shot Learning
We introduce compositional soft prompting (CSP), a parameter-efficient learning technique to improve the zero-shot compositionality of large-scale pretrained vision-language models (VLMs) like CLIP. We develop CSP for co…
AttributeCompositional Zero-Shot LearningObjectZero-Shot LearningAttention Based Simple Primitives for Open World Compositional Zero-Shot Learning
Compositional Zero-Shot Learning (CZSL) aims to predict unknown compositions made up of attribute and object pairs. Predicting compositions unseen during training is a challenging task. We are exploring Open World Compos…
AttributeCompositional Zero-Shot LearningObjectZero-Shot LearningBeyond Seen Primitive Concepts and Attribute-Object Compositional Learning
Learning from seen attribute-object pairs to generalize to unseen compositions has been studied extensively in Compositional Zero-Shot Learning (CZSL). However CZSL setup is still limited to seen attributes and objec…
AttributeCompositional Zero-Shot LearningZero-Shot LearningMAC: A Benchmark for Multiple Attributes Compositional Zero-Shot Learning
Compositional Zero-Shot Learning (CZSL) aims to learn semantic primitives (attributes and objects) from seen compositions and recognize unseen attribute-object compositions. Existing CZSL datasets focus on single attribu…
AttributeCompositional Zero-Shot LearningZero-Shot LearningLearning Conditional Attributes for Compositional Zero-Shot Learning
Compositional Zero-Shot Learning (CZSL) aims to train models to recognize novel compositional concepts based on learned concepts such as attribute-object combinations. One of the challenges is to model attributes interac…
AttributeCompositional Zero-Shot LearningZero-Shot Learning