Research on Patch Attentive Neural Process

2022-01-29 · Xiaohan Yu, Shaochen Mao

Attentive Neural Process (ANP) improves the fitting ability of Neural Process (NP) and improves its prediction accuracy, but the higher time complexity of the model imposes a limitation on the length of the input sequence. Inspired by models such as Vision Transformer (ViT) and Masked Auto-Encoder (MAE), we propose Patch Attentive Neural Process (PANP) using image patches as input and improve the structure of deterministic paths based on ANP, which allows the model to extract image features more accurately and efficiently reconstruction.

📄 PDF Abstract BibTeX arXiv:2202.01884

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Residual Connection 설명 없음

Efficient Vision-and-Language Pre-training with Text-Relevant Image Patch Selection

2024-01-11 · Wei Ye, Chaoya Jiang, Haiyang Xu, Chenhao Ye 외

Vision Transformers (ViTs) have become increasingly popular in large-scale Vision and Language Pre-training (VLP) models. Although previous VLP research has demonstrated the efficacy of ViTs, these efforts still struggle…

Attentive Symmetric Autoencoder for Brain MRI Segmentation

2022-09-19 · Junjia Huang, Haofeng Li, Guanbin Li, Xiang Wan

Self-supervised learning methods based on image patch reconstruction have witnessed great success in training auto-encoders, whose pre-trained weights can be transferred to fine-tune other downstream tasks of image under…

Image Segmentation Medical Image Segmentation MRI segmentation Segmentation+2

Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring

2020-04-11 · CVPR 2020 6 · Maitreya Suin, Kuldeep Purohit, A. N. Rajagopalan

This paper tackles the problem of motion deblurring of dynamic scenes. Although end-to-end fully convolutional designs have recently advanced the state-of-the-art in non-uniform motion deblurring, their performance-compl…

Deblurring Image Deblurring

Dynamic Attentive Graph Learning for Image Restoration

2021-09-14 · ICCV 2021 10 · Chong Mou, Jian Zhang, Zhuoyuan Wu

Non-local self-similarity in natural images has been verified to be an effective prior for image restoration. However, most existing deep non-local methods assign a fixed number of neighbors for each query item, neglecti…

Demosaicking Denoising Graph Learning Image Denoising+1

Vision Transformer with Attentive Pooling for Robust Facial Expression Recognition

2022-12-11 · Fanglei Xue, Qiangchang Wang, Zichang Tan, Zhongsong Ma 외

Facial Expression Recognition (FER) in the wild is an extremely challenging task. Recently, some Vision Transformers (ViT) have been explored for FER, but most of them perform inferiorly compared to Convolutional Neural …

Facial Expression Recognition Facial Expression Recognition (FER)Inductive Bias