Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information
How do neural language models keep track of number agreement between subject and verb? We show that `diagnostic classifiers', trained to predict number from the internal states of a language model, provide a detailed understanding of how, when, and where this information is represented. Moreover, they give us insight into when and where number information is corrupted in cases where the language model ends up making agreement errors. To demonstrate the causal role played by the representations we find, we then use agreement information to influence the course of the LSTM during the processing of difficult sentences. Results from such an intervention reveal a large increase in the language model's accuracy. Together, these results show that diagnostic classifiers give us an unrivalled detailed look into the representation of linguistic information in neural models, and demonstrate that this knowledge can be used to improve their performance.
Code (0)
등록된 구현이 없습니다.
Tasks
DiagnosticLanguage ModelingLanguage ModellingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Are Generative Classifiers More Robust to Adversarial Attacks?
There is a rising interest in studying the robustness of deep neural network classifiers against adversaries, with both advanced attack and defence techniques being actively developed. However, most recent work focuses o…
Adversarial DefenseAdversarial RobustnessMulti-modal Point-of-Care Diagnostics for COVID-19 Based On Acoustics and Symptoms
The research direction of identifying acoustic bio-markers of respiratory diseases has received renewed interest following the onset of COVID-19 pandemic. In this paper, we design an approach to COVID-19 diagnostic using…
DiagnosticVisualisation and 'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure
We investigate how neural networks can learn and process languages with hierarchical, compositional semantics. To this end, we define the artificial task of processing nested arithmetic expressions, and study whether dif…
DiagnosticDiagnosing model misspecification and performing generalized Bayes' updates via probabilistic classifiers
Model misspecification is a long-standing enigma of the Bayesian inference framework as posteriors tend to get overly concentrated on ill-informed parameter values towards the large sample limit. Tempering of the likelih…
Bayesian InferenceDiagnosticLikelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses
Convolutional Neural Networks have been shown to be vulnerable to adversarial examples, which are known to locate in subspaces close to where normal data lies but are not naturally occurring and of low probability. In th…
Adversarial DefenseAdversarial Robustness