Improving Few-Shot Image Classification Using Machine- and User-Generated Natural Language Descriptions
Humans can obtain the knowledge of novel visual concepts from language descriptions, and we thus use the few-shot image classification task to investigate whether a machine learning model can have this capability. Our proposed model, LIDE (Learning from Image and DEscription), has a text decoder to generate the descriptions and a text encoder to obtain the text representations of machine- or user-generated descriptions. We confirmed that LIDE with machine-generated descriptions outperformed baseline models. Moreover, the performance was improved further with high-quality user-generated descriptions. The generated descriptions can be viewed as the explanations of the model's predictions, and we observed that such explanations were consistent with prediction results. We also investigated why the language description improved the few-shot image classification performance by comparing the image representations and the text representations in the feature spaces.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderFew-Shot Image Classificationimage-classificationImage ClassificationSimilar Papers 제목 키워드 기반
Design of a Graphical User Interface for Few-Shot Machine Learning Classification of Electron Microscopy Data
The recent growth in data volumes produced by modern electron microscopes requires rapid, scalable, and flexible approaches to image segmentation and analysis. Few-shot machine learning, which can richly classify images …
Few-Shot LearningImage SegmentationSemantic SegmentationWill It Zero-Shot?: Predicting Zero-Shot Classification Performance For Arbitrary Queries
Vision-Language Models like CLIP create aligned embedding spaces for text and images, making it possible for anyone to build a visual classifier by simply naming the classes they want to distinguish. However, a model tha…
Convolutional Neural Networks for User Identificationbased on Motion Sensors Represented as Image
In this paper, we propose a deep learning approach for smartphone user identification based on analyzing motion signals recorded by the accelerometer and the gyroscope, during a single tap gesture performed by the user o…
General ClassificationUser IdentificationMasonTigers at SemEval-2024 Task 8: Performance Analysis of Transformer-based Models on Machine-Generated Text Detection
This paper presents the MasonTigers entry to the SemEval-2024 Task 8 - Multigenerator, Multidomain, and Multilingual Black-Box Machine-Generated Text Detection. The task encompasses Binary Human-Written vs. Machine-Gener…
Sentencetext-classificationText ClassificationText DetectionStable Attribute Group Editing for Reliable Few-shot Image Generation
Few-shot image generation aims to generate data of an unseen category based on only a few samples. Apart from basic content generation, a bunch of downstream applications hopefully benefit from this task, such as low-dat…
AttributeClassificationImage Generation