Interpreting Neural CWI Classifiers' Weights as Vocabulary Size
Complex Word Identification (CWI) is a task for the identification of words that are challenging for second-language learners to read. Even though the use of neural classifiers is now common in CWI, the interpretation of their parameters remains difficult. This paper analyzes neural CWI classifiers and shows that some of their parameters can be interpreted as vocabulary size. We present a novel formalization of vocabulary size measurement methods that are practiced in the applied linguistics field as a kind of neural classifier. We also contribute to building a novel dataset for validating vocabulary testing and readability via crowdsourcing.
Code (0)
등록된 구현이 없습니다.
Tasks
Complex Word IdentificationSimilar Papers 제목 키워드 기반
Disentangling MLP Neuron Weights in Vocabulary Space
Interpreting the information encoded in language model weights remains a fundamental challenge in mechanistic interpretability. In this work, we introduce ROTATE (Rotation-Optimized Token Alignment in weighT spacE), a da…
Finding the Optimal Vocabulary Size for Neural Machine Translation
We cast neural machine translation (NMT) as a classification task in an autoregressive setting and analyze the limitations of both classification and autoregression components. Classifiers are known to perform better wit…
ClassificationGeneral ClassificationMachine TranslationNMT+1Initial Findings on Sensor based Open Vocabulary Activity Recognition via Text Embedding Inversion
Conventional human activity recognition (HAR) relies on classifiers trained to predict discrete activity classes, inherently limiting recognition to activities explicitly present in the training set. Such classifiers wou…
Activity RecognitionDescriptiveHuman Activity RecognitionPrompt EngineeringGenetic Algorithms For Extractive Summarization
Most current work in NLP utilizes deep learning, which requires a lot of training data and computational power. This paper investigates the strengths of Genetic Algorithms (GAs) for extractive summarization, as we hypoth…
Deep LearningExtractive SummarizationSentenceDetecting Twenty-thousand Classes using Image-level Supervision
Current object detectors are limited in vocabulary size due to the small scale of detection datasets. Image classifiers, on the other hand, reason about much larger vocabularies, as their datasets are larger and easier t…
Cross-Domain Few-Shot Object Detectionimage-classificationImage ClassificationOpen Vocabulary Object Detection