AILAB-Udine@SMM4H 22: Limits of Transformers and BERT Ensembles

2022-09-07 · Beatrice Portelli, Simone Scaboro, Emmanuele Chersoni, Enrico Santus, Giuseppe Serra

This paper describes the models developed by the AILAB-Udine team for the SMM4H 22 Shared Task. We explored the limits of Transformer based models on text classification, entity extraction and entity normalization, tackling Tasks 1, 2, 5, 6 and 10. The main take-aways we got from participating in different tasks are: the overwhelming positive effects of combining different architectures when using ensemble learning, and the great potential of generative models for term normalization.

📄 PDF Abstract BibTeX arXiv:2209.03452

Code (0)

등록된 구현이 없습니다.

Tasks

Ensemble Learning text-classification Text Classification

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Adam 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Multi-Head Attention 설명 없음

Residual Connection 설명 없음

AILAB-Udine@SMM4H’22: Limits of Transformers and BERT Ensembles

2022-10-01 · SMM4H (COLING) 2022 10 · Beatrice Portelli, Simone Scaboro, Emmanuele Chersoni, Enrico Santus 외

This paper describes the models developed by the AILAB-Udine team for the SMM4H’22 Shared Task. We explored the limits of Transformer based models on text classification, entity extraction and entity normalization, tackl…

Ensemble Learning text-classification Text Classification

An unambiguous cloudiness index for nonwovens

2022-01-06 · Michael Godehardt, Ali Moghiseh, Christine Oetjen, Joachim Ohser 외

Cloudiness or formation is a concept routinely used in industry to address deviations from homogeneity in nonwovens and papers. Measuring a cloudiness index based on image data is a common task in industrial quality assu…

Detection of depression on social networks using transformers and ensembles

2023-05-09 · Ilija Tavchioski, Marko Robnik-Šikonja, Senja Pollak

As the impact of technology on our lives is increasing, we witness increased use of social media that became an essential tool not only for communication but also for sharing information with community about our thoughts…

Depression Detection Language Modeling Language Modelling Transfer Learning

Assessing cloudiness in nonwovens

2022-04-13 · Michael Godehardt, Ali Moghiseh, Christine Oetjen, Joachim Ohser 외

The homogeneity of filter media is important for material selection and quality control, along with the specific weight (nominal grammage) and the distribution of the local weight. Cloudiness or formation is a concept us…

A method for Cloud Mapping in the Field of View of the Infra-Red Camera during the EUSO-SPB1 flight

2019-09-12 · Alessandro Bruno, Anna Anzalone, Carlo Vigorito

EUSO-SPB1 was released on April 24th, 2017, from the NASA balloon launch site in Wanaka (New Zealand) and landed on the South Pacific Ocean on May 7th. The data collected by the instruments onboard the balloon were analy…

Image Enhancement