Tailoring Visual Object Representations to Human Requirements: A Case Study with a Recycling Robot
Robots are well-suited to alleviate the burden of repetitive and tedious manipulation tasks. In many applications, though, a robot may be asked to interact with a wide variety of objects, making it hard or even impossible to pre-program visual object classifiers suitable for the task of interest. In this work, we study the problem of learning a classifier for visual objects based on a few examples provided by humans. We frame this problem from the perspective of learning a suitable visual object representation that allows us to distinguish the desired object category from others. Our proposed approach integrates human supervision into the representation learning process by combining contrastive learning with an additional loss function that brings the representations of human examples close to each other in the latent space. Our experiments show that our proposed method performs better than self-supervised and fully supervised learning methods in offline evaluations and can also be used in real-time by a robot in a simplified recycling domain, where recycling streams contain a variety of objects.
Code (1)
Tasks
Contrastive LearningObjectRepresentation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction time
From CNNs to attention mechanisms, encoding inductive biases into neural networks has been a fruitful source of improvement in machine learning. Adding auxiliary losses to the main objective function is a general way of …
Inductive BiasMeta-LearningTransductive LearningTowards a Framework for Visual Intelligence in Service Robotics: Epistemic Requirements and Gap Analysis
A key capability required by service robots operating in real-world, dynamic environments is that of Visual Intelligence, i.e., the ability to use their vision system, reasoning components and background knowledge to mak…
Object RecognitionClothHMR: 3D Mesh Recovery of Humans in Diverse Clothing from Single Image
With 3D data rapidly emerging as an important form of multimedia information, 3D human mesh recovery technology has also advanced accordingly. However, current methods mainly focus on handling humans wearing tight clothi…
Human Mesh RecoveryInstruction-Driven Fusion of Infrared-Visible Images: Tailoring for Diverse Downstream Tasks
The primary value of infrared and visible image fusion technology lies in applying the fusion results to downstream tasks. However, existing methods face challenges such as increased training complexity and significantly…
Infrared And Visible Image Fusionobject-detectionObject DetectionSalient Object Detection+2Text-to-Sticker: Style Tailoring Latent Diffusion Models for Human Expression
We introduce Style Tailoring, a recipe to finetune Latent Diffusion Models (LDMs) in a distinct domain with high visual quality, prompt alignment and scene diversity. We choose sticker image generation as the target doma…
DiversityImage GenerationPrompt Engineering