Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching
Natural language processing (NLP) models trained on people-generated data can be unreliable because, without any constraints, they can learn from spurious correlations that are not relevant to the task. We hypothesize that enriching models with speaker information in a controlled, educated way can guide them to pick up on relevant inductive biases. For the speaker-driven task of predicting code-switching points in English--Spanish bilingual dialogues, we show that adding sociolinguistically-grounded speaker features as prepended prompts significantly improves accuracy. We find that by adding influential phrases to the input, speaker-informed models learn useful and explainable linguistic information. To our knowledge, we are the first to incorporate speaker characteristics in a neural model for code-switching, and more generally, take a step towards developing transparent, personalized models that use speaker information in a controlled way.
Code (1)
Similar Papers 제목 키워드 기반
Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching
Natural language processing (NLP) models trained on people-generated data can be unreliable because, without any constraints, they can learn from spurious correlations or propagate dangerous biases about personal identit…
Biases for Emergent Communication in Multi-agent Reinforcement Learning
We study the problem of emergent communication, in which language arises because speakers and listeners must communicate information in order to solve tasks. In temporally extended reinforcement learning domains, it has …
Multi-agent Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Eliminating Inductive Bias in Reward Models with Information-Theoretic Guidance
Reward models (RMs) are essential in reinforcement learning from human feedback (RLHF) to align large language models (LLMs) with human values. However, RM training data is commonly recognized as low-quality, containing …
Reinforcement LearningMUSA: Multi-lingual Speaker Anonymization via Serial Disentanglement
Speaker anonymization is an effective privacy protection solution designed to conceal the speaker's identity while preserving the linguistic content and para-linguistic information of the original speech. While most prio…
DisentanglementSpeaker anonymization“A Little Birdie Told Me ... ” - Inductive Biases for Rumour Stance Detection on Social Media
The rise in the usage of social media has placed it in a central position for news dissemination and consumption. This greatly increases the potential for proliferation of rumours and misinformation. In an effort to miti…
MisinformationPositionRumour DetectionStance Detection