Boosting Transformers for Job Expression Extraction and Classification in a Low-Resource Setting

2021-09-17 · Lukas Lange, Heike Adel, Jannik Strötgen

In this paper, we explore possible improvements of transformer models in a low-resource setting. In particular, we present our approaches to tackle the first two of three subtasks of the MEDDOPROF competition, i.e., the extraction and classification of job expressions in Spanish clinical texts. As neither language nor domain experts, we experiment with the multilingual XLM-R transformer model and tackle these low-resource information extraction tasks as sequence-labeling problems. We explore domain- and language-adaptive pretraining, transfer learning and strategic datasplits to boost the transformer model. Our results show strong improvements using these methods by up to 5.3 F1 points compared to a fine-tuned XLM-R model. Our best models achieve 83.2 and 79.3 F1 for the first two tasks, respectively.

📄 PDF Abstract BibTeX arXiv:2109.08597

Code (0)

등록된 구현이 없습니다.

Tasks

Transfer LearningXLM-R

Methods 이 논문이 사용한 방법론

XLM-R XLM-R

Tricks and Plug-ins for Gradient Boosting with Transformers

2025-08-04 · Biyi Fang, Truong Vo, Jean Utke, Diego Klabjan arxiv

Transformer architectures dominate modern NLP but often demand heavy computational resources and intricate hyperparameter tuning. To mitigate these challenges, we propose a novel framework, BoostTransformer, that augment…

Text Classification

AU-Supervised Convolutional Vision Transformers for Synthetic Facial Expression Recognition

2022-07-20 · Shuyi Mao, Xinpeng Li, Junyao Chen, Xiaojiang Peng

The paper describes our proposed methodology for the six basic expression classification track of Affective Behavior Analysis in-the-wild (ABAW) Competition 2022. In Learing from Synthetic Data(LSD) task, facial expressi…

Face Recognition Facial Expression Recognition Facial Expression Recognition (FER)

BanglaASTE: A Novel Framework for Aspect-Sentiment-Opinion Extraction in Bangla E-commerce Reviews Using Ensemble Deep Learning

2025-11-26 · Ariful Islam, Md Rifat Hossen, Abir Ahmed, B M Taslimul Haque arxiv

Aspect-Based Sentiment Analysis (ABSA) has emerged as a critical tool for extracting fine-grained sentiment insights from user-generated content, particularly in e-commerce and social media domains. However, research on …

Aspect Sentiment Triplet Extraction Semantic Similarity Sentiment Analysis

Boosting Hyperspectral Image Classification with Gate-Shift-Fuse Mechanisms in a Novel CNN-Transformer Approach

2024-06-20 · Mohamed Fadhlallah Guerri, Cosimo Distante, Paolo Spagnolo, Fares Bougourzi 외

During the process of classifying Hyperspectral Image (HSI), every pixel sample is categorized under a land-cover type. CNN-based techniques for HSI classification have notably advanced the field by their adept feature r…

Hyperspectral Image Classification image-classification Image Classification

Boosting Continuous Emotion Recognition with Self-Pretraining using Masked Autoencoders, Temporal Convolutional Networks, and Transformers

2024-03-18 · Weiwei Zhou, Jiada Lu, Chenkun Ling, Weifeng Wang 외

Human emotion recognition holds a pivotal role in facilitating seamless human-computer interaction. This paper delineates our methodology in tackling the Valence-Arousal (VA) Estimation Challenge, Expression (Expr) Class…

Emotion Recognition