Prototype memory and attention mechanisms for few shot image generation

2021-09-29 · ICLR 2022 4 · Tianqin Li, Zijie Li, Andrew Luo, Harold Rockwell, Amir Barati Farimani, Tai Sing Lee

Recent discoveries indicate that the neural codes in the primary visual cortex (V1) of macaque monkeys are complex, diverse and sparse. This leads us to ponder the computational advantages and functional role of these “grandmother cells." Here, we propose that such cells can serve as prototype memory priors that bias and shape the distributed feature processing within the image generation process in the brain. These memory prototypes are learned by momentum online clustering and are utilized via a memory-based attention operation, which we define as Memory Concept Attention (MoCA). To test our proposal, we show in a few-shot image generation task, that having a prototype memory during attention can improve image synthesis quality, learn interpretable visual concept clusters, as well as improve the robustness of the model. Interestingly, we also find that our attentional memory mechanism can implicitly modify the horizontal connections by updating the transformation into the prototype embedding space for self-attention. Insofar as GANs can be seen as plausible models for reasoning about the top-down synthesis in the analysis-by-synthesis loop of the hierarchical visual cortex, our findings demonstrate a plausible computational role for these “prototype concept" neurons in visual processing in the brain.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Image Generation Online Clustering

Methods 이 논문이 사용한 방법론

Test 설명 없음

Learning to Learn Variational Semantic Memory

2020-10-20 · NeurIPS 2020 12 · XianTong Zhen, Yingjun Du, Huan Xiong, Qiang Qiu 외

In this paper, we introduce variational semantic memory into meta-learning to acquire long-term knowledge for few-shot learning. The variational semantic memory accrues and stores semantic information for the probabilist…

Few-Shot Learning General Knowledge Meta-Learning Variational Inference

Multi-Label Few-Shot Learning for Aspect Category Detection

2021-05-29 · ACL 2021 5 · Mengting Hu, Shiwan Zhao, Honglei Guo, Chao Xue 외

Aspect category detection (ACD) in sentiment analysis aims to identify the aspect categories mentioned in a sentence. In this paper, we formulate ACD in the few-shot learning scenario. However, existing few-shot learning…

Aspect Category Detection Few-Shot Learning Sentence Sentiment Analysis

Attention-Based Prototype Calibration for Multi-Rater Few-Shot Medical Image Segmentation

2026-06-15 · Truong Vu, Minh Khoi Ho, Yutong Xie arxiv

Few-shot medical image segmentation methods typically assume a single ground-truth annotation, overlooking systematic variability across expert raters commonly observed in clinical datasets. We propose an attention-based…

Medical Image Segmentation Personalized Segmentation

Channel-Spatial-Based Few-Shot Bird Sound Event Detection

2023-06-18 · Lingwen Liu, Yuxuan Feng, Haitao Fu, Yajie Yang 외

In this paper, we propose a model for bird sound event detection that focuses on a small number of training samples within the everyday long-tail distribution. As a result, we investigate bird sound detection using the f…

Event Detection Few-Shot Learning Sound Classification Sound Event Detection

Few-Shot Object Detection by Attending to Per-Sample-Prototype

2021-09-16 · Hojun Lee, Myunggi Lee, Nojun Kwak

Few-shot object detection aims to detect instances of specific categories in a query image with only a handful of support samples. Although this takes less effort than obtaining enough annotated images for supervised obj…

Few-Shot Object Detection Meta-Learning Object object-detection+1