Evaluating Contextualized Language Models for Hungarian

2021-02-22 · Judit Ács, Dániel Lévai, Dávid Márk Nemeskey, András Kornai

We present an extended comparison of contextualized language models for Hungarian. We compare huBERT, a Hungarian model against 4 multilingual models including the multilingual BERT model. We evaluate these models through three tasks, morphological probing, POS tagging and NER. We find that huBERT works better than the other models, often by a large margin, particularly near the global optimum (typically at the middle layers). We also find that huBERT tends to generate fewer subwords for one word and that using the last subword for token-level tasks is generally a better choice than using the first one.

📄 PDF Abstract BibTeX arXiv:2102.10848

Code (1)

juditacs/hubert_eval 공식 구현 pytorch

Tasks

NERPOSPOS Tagging

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Adam 설명 없음

Multi-Head Attention 설명 없음

Residual Connection 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

WordPiece 설명 없음

OpenHuEval: Evaluating Large Language Model on Hungarian Specifics

2025-03-27 · Haote Yang, Xingjian Wei, Jiang Wu, Noémi Ligeti-Nagy 외

We introduce OpenHuEval, the first benchmark for LLMs focusing on the Hungarian language and specifics. OpenHuEval is constructed from a vast collection of Hungarian-specific materials sourced from multiple origins. In t…

Language Modeling Language Modelling Large Language Model Model Optimization

A Comparative Analysis of Static Word Embeddings for Hungarian

2025-05-12 · Máté Gedeon

This paper presents a comprehensive analysis of various static word embeddings for Hungarian, including traditional models such as Word2Vec, FastText, as well as static embeddings derived from BERT-based models using dif…

named-entity-recognition Named Entity Recognition Named Entity Recognition (NER)NER+4

Universal Dependencies and Morphology for Hungarian - and on the Price of Universality

2017-04-01 · EACL 2017 4 · Veronika Vincze, Katalin Simk{\'o}, Zsolt Sz{\'a}nt{\'o}, Rich{\'a}rd Farkas

In this paper, we present how the principles of universal dependencies and morphology have been adapted to Hungarian. We report the most challenging grammatical phenomena and our solutions to those. On the basis of the a…

Morphological Tagging

Developing neural machine translation models for Hungarian-English

2021-11-07 · Attila Nagy

I train models for the task of neural machine translation for English-Hungarian and Hungarian-English, using the Hunglish2 corpus. The main contribution of this work is evaluating different data augmentation methods duri…

Data Augmentation Dependency Parsing Machine Translation NMT+1

On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations

2022-03-25 · ACL 2022 5 · Yang Trista Cao, Yada Pruksachatkun, Kai-Wei Chang, Rahul Gupta 외

Multiple metrics have been introduced to measure fairness in various natural language processing tasks. These metrics can be roughly categorized into two categories: 1) \emph{extrinsic metrics} for evaluating fairness in…

Fairness