Performance in a dialectal profiling task of LLMs for varieties of Brazilian Portuguese
Different of biases are reproduced in LLM-generated responses, including dialectal biases. A study based on prompt engineering was carried out to uncover how LLMs discriminate varieties of Brazilian Portuguese, specifically if sociolinguistic rules are taken into account in four LLMs: GPT 3.5, GPT-4o, Gemini, and Sabi.-2. The results offer sociolinguistic contributions for an equity fluent NLP technology.
Code (0)
등록된 구현이 없습니다.
Tasks
Prompt EngineeringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Guidelines and Annotation Framework for Arabic Author Profiling
In this paper, we present the annotation pipeline and the guidelines we wrote as part of an effort to create a large manually annotated Arabic author profiling dataset from various social media sources covering 16 Arabic…
Author ProfilingData-Augmentation-Based Dialectal Adaptation for LLMs
This report presents GMUNLP's participation to the Dialect-Copa shared task at VarDial 2024, which focuses on evaluating the commonsense reasoning capabilities of large language models (LLMs) on South Slavic micro-dialec…
Data AugmentationNatural Language UnderstandingDialectal Toxicity Detection: Evaluating LLM-as-a-Judge Consistency Across Language Varieties
There has been little systematic study on how dialectal differences affect toxicity detection by modern LLMs. Furthermore, although using LLMs as evaluators ("LLM-as-a-judge") is a growing research area, their sensitivit…
GRDD+: An Extended Greek Dialectal Dataset with Cross-Architecture Fine-tuning Evaluation
We present an extended Greek Dialectal Dataset (GRDD+) 1that complements the existing GRDD dataset with more data from Cretan, Cypriot, Pontic and Northern Greek, while we add six new varieties: Greco-Corsican, Griko (So…
AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic
Dialectal Arabic (DA) varieties are under-served by language technologies, particularly large language models (LLMs). This trend threatens to exacerbate existing social inequalities and limits LLM applications, yet the r…
Language Modelling