Ensemble Making Few-Shot Learning Stronger

2021-05-12 · Qing Lin, Yongbin Liu, Wen Wen, Zhihua Tao

Few-shot learning has been proposed and rapidly emerging as a viable means for completing various tasks. Many few-shot models have been widely used for relation learning tasks. However, each of these models has a shortage of capturing a certain aspect of semantic features, for example, CNN on long-range dependencies part, Transformer on local features. It is difficult for a single model to adapt to various relation learning, which results in the high variance problem. Ensemble strategy could be competitive on improving the accuracy of few-shot relation extraction and mitigating high variance risks. This paper explores an ensemble approach to reduce the variance and introduces fine-tuning and feature attention strategies to calibrate relation-level features. Results on several few-shot relation learning tasks show that our model significantly outperforms the previous state-of-the-art models.

📄 PDF Abstract BibTeX arXiv:2105.11904

Code (0)

등록된 구현이 없습니다.

Tasks

Few-Shot LearningRelationRelation Extraction

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Residual Connection 설명 없음

Multi-Head Attention 설명 없음

Adam 설명 없음

Time Adaptive Reinforcement Learning

2020-04-18 · Chris Reinke

Reinforcement learning (RL) allows to solve complex tasks such as Go often with a stronger performance than humans. However, the learned behaviors are usually fixed to specific tasks and unable to adapt to different cont…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Prompt Amplification and Zero-Shot Late Fusion in Audio-Language Models for Speech Emotion Recognition

2026-03-24 · Saurabh Kataria, Xiao Hu arxiv

Audio-Language Models (ALMs) are making strides in understanding speech and non-speech audio. However, domain-specialist Foundation Models (FMs) remain the best for closed-ended speech processing tasks such as Speech Emo…

Speech Emotion Recognition

Few-shot Learning via Dirichlet Tessellation Ensemble

2021-09-29 · ICLR 2022 4 · Chunwei Ma, Ziyun Huang, Mingchen Gao, Jinhui Xu

Few-shot learning (FSL) is the process of rapid generalization from abundant base samples to inadequate novel samples. Despite extensive research in recent years, FSL is still not yet able to generate satisfactory soluti…

Few-Shot Learning

Few-shot Learning as Cluster-induced Voronoi Diagrams: A Geometric Approach

2022-02-05 · Chunwei Ma, Ziyun Huang, Mingchen Gao, Jinhui Xu

Few-Shot Learning

To Stay or Not to Stay in the Pre-train Basin: Insights on Ensembling in Transfer Learning

2023-03-06 · NeurIPS 2023 11 · Ildus Sadrtdinov, Dmitrii Pozdeev, Dmitry Vetrov, Ekaterina Lobacheva

Transfer learning and ensembling are two popular techniques for improving the performance and robustness of neural networks. Due to the high cost of pre-training, ensembles of models fine-tuned from a single pre-trained …

Diversity Transfer Learning