Does a Neural Network Really Encode Symbolic Concepts?
Recently, a series of studies have tried to extract interactions between input variables modeled by a DNN and define such interactions as concepts encoded by the DNN. However, strictly speaking, there still lacks a solid guarantee whether such interactions indeed represent meaningful concepts. Therefore, in this paper, we examine the trustworthiness of interaction concepts from four perspectives. Extensive empirical studies have verified that a well-trained DNN usually encodes sparse, transferable, and discriminative concepts, which is partially aligned with human intuition.
Code (1)
Similar Papers 제목 키워드 기반
Can the Inference Logic of Large Language Models be Disentangled into Symbolic Concepts?
In this paper, we explain the inference logic of large language models (LLMs) as a set of symbolic concepts. Many recent studies have discovered that traditional DNNs usually encode sparse symbolic concepts. However, bec…
SentenceLogical Interpretations of Autoencoders
The unification of low-level perception and high-level reasoning is a long-standing problem in artificial intelligence, which has the potential to not only bring the areas of logic and learning closer together but also d…
Deep LearningDimensionality ReductionWhere We Have Arrived in Proving the Emergence of Sparse Symbolic Concepts in AI Models
This study aims to prove the emergence of symbolic concepts (or more precisely, sparse primitive inference patterns) in well-trained deep neural networks (DNNs). Specifically, we prove the following three conditions for …
Closed-Form Interpretation of Neural Network Latent Spaces with Symbolic Gradients
It has been demonstrated in many scientific fields that artificial neural networks like autoencoders or Siamese networks encode meaningful concepts in their latent spaces. However, there does not exist a comprehensive fr…
FormTowards Supporting Legal Argumentation with NLP: Is More Data Really All You Need?
Modeling legal reasoning and argumentation justifying decisions in cases has always been central to AI & Law, yet contemporary developments in legal NLP have increasingly focused on statistically classifying legal conclu…
AllLegal Reasoning