Shaping Visual Representations with Language for Few-shot Classification

2019-11-06 · ACL 2020 6 · Jesse Mu, Percy Liang, Noah Goodman

By describing the features and abstractions of our world, language is a crucial tool for human learning and a promising source of supervision for machine learning models. We use language to improve few-shot visual classification in the underexplored scenario where natural language task descriptions are available during training, but unavailable for novel tasks at test time. Existing models for this setting sample new descriptions at test time and use those to classify images. Instead, we propose language-shaped learning (LSL), an end-to-end model that regularizes visual representations to predict language. LSL is conceptually simpler, more data efficient, and outperforms baselines in two challenging few-shot domains.

📄 PDF Abstract BibTeX arXiv:1911.02683

Code (2)

jayelm/lsl 공식 구현 pytorch

https://worksheets.codalab.org/worksheets/0x55ed347e3beb4fa6971d3a226e48fc92 공식 구현

Tasks

Classification General Classification Meta-Learning Representation Learning

Methods 이 논문이 사용한 방법론

Test 설명 없음

Shaping Visual Representations with Attributes for Few-Shot Recognition

2021-12-13 · Haoxing Chen, Huaxiong Li, Yaohui Li, Chunlin Chen

Few-shot recognition aims to recognize novel categories under low-data regimes. Some recent few-shot recognition methods introduce auxiliary semantic modality, i.e., category attribute information, into representation le…

Attribute Few-Shot Learning Metric Learning Representation Learning

Improved Few-Shot Image Classification Through Multiple-Choice Questions

2024-07-23 · Dipika Khullar, Emmett Goodman, Negin Sokhandan

Through a simple multiple choice language prompt a VQA model can operate as a zero-shot image classifier, producing a classification label. Compared to typical image encoders, VQA models offer an advantage: VQA-produced …

Articles Few-Shot Image Classification image-classification Image Classification+2

Exploiting Category Names for Few-Shot Classification with Vision-Language Models

2022-11-29 · Taihong Xiao, ZiRui Wang, Liangliang Cao, Jiahui Yu 외

Vision-language foundation models pretrained on large-scale data provide a powerful tool for many visual understanding tasks. Notably, many vision-language models build two encoders (visual and textual) that can map two …

Classification Few-Shot Image Classification image-classification Image Classification

Improving Few-Shot Image Classification Using Machine- and User-Generated Natural Language Descriptions

2022-07-07 · Findings (NAACL) 2022 7 · Kosuke Nishida, Kyosuke Nishida, Shuichi Nishioka

Humans can obtain the knowledge of novel visual concepts from language descriptions, and we thus use the few-shot image classification task to investigate whether a machine learning model can have this capability. Our pr…

Decoder Few-Shot Image Classification image-classification Image Classification

On the Limits of Multi-modal Meta-Learning with Auxiliary Task Modulation Using Conditional Batch Normalization

2024-05-29 · Jordi Armengol-Estapé, Vincent Michalski, Ramnath Kumar, Pierre-Luc St-Charles 외

Few-shot learning aims to learn representations that can tackle novel tasks given a small number of examples. Recent studies show that cross-modal learning can improve representations for few-shot classification. More sp…

Few-Shot Learning Meta-Learning