From Words to Worlds: Compositionality for Cognitive Architectures
Large language models (LLMs) are very performant connectionist systems, but do they exhibit more compositionality? More importantly, is that part of why they perform so well? We present empirical analyses across four LLM families (12 models) and three task categories, including a novel task introduced below. Our findings reveal a nuanced relationship in learning of compositional strategies by LLMs -- while scaling enhances compositional abilities, instruction tuning often has a reverse effect. Such disparity brings forth some open issues regarding the development and improvement of large language models in alignment with human cognitive capacities.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Design for a Darwinian Brain: Part 1. Philosophy and Neuroscience
Physical symbol systems are needed for open-ended cognition. A good way to understand physical symbol systems is by comparison of thought to chemistry. Both have systematicity, productivity and compositionality. The stat…
PhilosophyAOGNets: Compositional Grammatical Architectures for Deep Learning
Neural architectures are the foundation for improving performance of deep neural networks (DNNs). This paper presents deep compositional grammatical architectures which harness the best of two worlds: grammar models and …
Adversarial DefenseDeep LearningImage Classificationobject-detection+3Analyzing Compositionality-Sensitivity of NLI Models
Success in natural language inference (NLI) should require a model to understand both lexical and compositional semantics. However, through adversarial evaluation, we find that several state-of-the-art models with divers…
DiagnosticNatural Language InferenceSensitivityBootstrapping Cognitive Agents with a Large Language Model
Large language models contain noisy general knowledge of the world, yet are hard to train or fine-tune. On the other hand cognitive architectures have excellent interpretability and are flexible to update but require a l…
General KnowledgeLanguage ModelingLanguage ModellingLarge Language ModelContextualized word senses: from attention to compositionality
The neural architectures of language models are becoming increasingly complex, especially that of Transformers, based on the attention mechanism. Although their application to numerous natural language processing tasks h…