Analyzing machine-learned representations: A natural language case study
As modern deep networks become more complex, and get closer to human-like capabilities in certain domains, the question arises of how the representations and decision rules they learn compare to the ones in humans. In this work, we study representations of sentences in one such artificial system for natural language processing. We first present a diagnostic test dataset to examine the degree of abstract composable structure represented. Analyzing performance on these diagnostic tests indicates a lack of systematicity in the representations and decision rules, and reveals a set of heuristic strategies. We then investigate the effect of the training distribution on learning these heuristic strategies, and study changes in these representations with various augmentations to the training set. Our results reveal parallels to the analogous representations in people. We find that these systems can learn abstract rules and generalize them to new contexts under certain circumstances -- similar to human zero-shot reasoning. However, we also note some shortcomings in this generalization behavior -- similar to human judgment errors like belief bias. Studying these parallels suggests new ways to understand psychological phenomena in humans as well as informs best strategies for building artificial intelligence with human-like language understanding.
Code (1)
Tasks
DiagnosticSimilar Papers 제목 키워드 기반
Predicting Foreign Language Usage from English-Only Social Media Posts
Social media is known for its multi-cultural and multilingual interactions, a natural product of which is code-mixing. Multilingual speakers mix languages they tweet to address a different audience, express certain feeli…
Cross-Lingual TransferLanguage IdentificationNative Language IdentificationTransfer LearningNatural Language Multitasking: Analyzing and Improving Syntactic Saliency of Hidden Representations
We train multi-task autoencoders on linguistic tasks and analyze the learned hidden sentence representations. The representations change significantly when translation and part-of-speech decoders are added. The more deco…
SentenceTranslationUsing Natural Language and Program Abstractions to Instill Human Inductive Biases in Machines
Strong inductive biases give humans the ability to quickly learn to perform a variety of tasks. Although meta-learning is a method to endow neural networks with useful inductive biases, agents trained by meta-learning ma…
Meta-LearningMeta Reinforcement LearningProgram inductionReinforcement Learning (RL)On the Linguistic Representational Power of Neural Machine Translation Models
Despite the recent success of deep neural networks in natural language processing (NLP), their interpretability remains a challenge. We analyze the representations learned by neural machine translation models at various …
Machine TranslationNMTTranslationOut-of-Order Decoding for Robust Neural Machine Translation
Autoregressive recurrent neural decoders that generate sequences of tokens one-by-one and left-to-right are the workhorse of modern machine translation. In this work, we propose a new decoder architecture that can genera…
DecoderMachine TranslationPositionTranslation