Fair Comparison between Efficient Attentions
Transformers have been successfully used in various fields and are becoming the standard tools in computer vision. However, self-attention, a core component of transformers, has a quadratic complexity problem, which limits the use of transformers in various vision tasks that require dense prediction. Many studies aiming at solving this problem have been reported proposed. However, no comparative study of these methods using the same scale has been reported due to different model configurations, training schemes, and new methods. In our paper, we validate these efficient attention models on the ImageNet1K classification task by changing only the attention operation and examining which efficient attention is better.
Code (1)
Similar Papers 제목 키워드 기반
Efficient Attentions for Long Document Summarization
The quadratic computational and memory complexities of large Transformers have limited their scalability for long document summarization. In this paper, we propose Hepos, a novel efficient encoder-decoder attention with …
DecoderDocument SummarizationQuantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics
Measuring bias is key for better understanding and addressing unfairness in NLP/ML models. This is often done via fairness metrics which quantify the differences in a model's behaviour across a range of demographic group…
FairnessStairway to Fairness: Connecting Group and Individual Fairness
Fairness in recommender systems (RSs) is commonly categorised into group fairness and individual fairness. However, there is no established scientific understanding of the relationship between the two fairness types, as …
A Comparison of Differential Performance Metrics for the Evaluation of Automatic Speaker Verification Fairness
When decisions are made and when personal data is treated by automated processes, there is an expectation of fairness -- that members of different demographic groups receive equitable treatment. This expectation applies …
Face RecognitionFairnessSpeaker VerificationFair Feature Selection: A Comparison of Multi-Objective Genetic Algorithms
Machine learning classifiers are widely used to make decisions with a major impact on people's lives (e.g. accepting or denying a loan, hiring decisions, etc). In such applications,the learned classifiers need to be both…
Fairnessfeature selection