Native Language Identification with Big Bird Embeddings
Native Language Identification (NLI) intends to classify an author's native language based on their writing in another language. Historically, the task has heavily relied on time-consuming linguistic feature engineering, and transformer-based NLI models have thus far failed to offer effective, practical alternatives. The current work investigates if input size is a limiting factor, and shows that classifiers trained using Big Bird embeddings outperform linguistic feature engineering models by a large margin on the Reddit-L2 dataset. Additionally, we provide further insight into input length dependencies, show consistent out-of-sample performance, and qualitatively analyze the embedding space. Given the effectiveness and computational efficiency of this method, we believe it offers a promising avenue for future NLI work.
Code (1)
Tasks
Computational EfficiencyFeature EngineeringLanguage IdentificationNative Language IdentificationSimilar Papers 제목 키워드 기반
Bird Vocalization Embedding Extraction Using Self-Supervised Disentangled Representation Learning
This paper addresses the extraction of the bird vocalization embedding from the whole song level using disentangled representation learning (DRL). Bird vocalization embeddings are necessary for large-scale bioacoustic ta…
Representation LearningA Bird Song Detector for improving bird identification through Deep Learning: a case study from Doñana
Passive Acoustic Monitoring is a key tool for biodiversity conservation, but the large volumes of unsupervised audio it generates present major challenges for extracting meaningful information. Deep Learning offers promi…
Hierarchical Re-Classification: Combining Animal Classification Models with Vision Transformers
State-of-the-art animal classification models like SpeciesNet provide predictions across thousands of species but use conservative rollup strategies, resulting in many animals labeled at high taxonomic levels rather than…
Metric LearningRecognizing Birds from Sound - The 2018 BirdCLEF Baseline System
Reliable identification of bird species in recorded audio files would be a transformative tool for researchers, conservation biologists, and birders. In recent years, artificial neural networks have greatly improved the …
BIG-bench Machine LearningBird Audio DetectionPerch 2.0 transfers 'whale' to underwater tasks
Perch 2.0 is a supervised bioacoustics foundation model pretrained on 14,597 species, including birds, mammals, amphibians, and insects, and has state-of-the-art performance on multiple benchmarks. Given that Perch 2.0 i…
Transfer Learning