On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers
A recent numerical study observed that neural network classifiers enjoy a large degree of symmetry in the penultimate layer. Namely, if $h(x) = Af(x) +b$ where $A$ is a linear map and $f$ is the output of the penultimate layer of the network (after activation), then all data points $x_{i, 1}, \dots, x_{i, N_i}$ in a class $C_i$ are mapped to a single point $y_i$ by $f$ and the points $y_i$ are located at the vertices of a regular $k-1$-dimensional standard simplex in a high-dimensional Euclidean space. We explain this observation analytically in toy models for highly expressive deep neural networks. In complementary examples, we demonstrate rigorously that even the final output of the classifier $h$ is not uniform over data samples from a class $C_i$ if $h$ is a shallow network (or if the deeper layers do not bring the data samples into a convenient geometric configuration).
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame
Neural Collapse (NC) is a recently observed phenomenon in neural networks that characterises the solution space of the final classifier layer when trained until zero training loss. Specifically, NC suggests that the fina…
Leveraging Intermediate Neural Collapse with Simplex ETFs for Efficient Deep Neural Networks
Neural collapse is a phenomenon observed during the terminal phase of neural network training, characterized by the convergence of network activations, class means, and linear classifier weights to a simplex equiangular …
E-TCAV: Formalizing Penultimate Proxies for Efficient Concept Based Interpretability
TCAV (Testing with Concept Activation Vectors) is an interpretability method that assesses the alignment between the internal representations of a trained neural network and human-understandable, high-level concepts. Tho…
Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity
There is a recently discovered and intriguing phenomenon called Neural Collapse: at the terminal phase of training a deep neural network for classification, the within-class penultimate feature means and the associated c…
Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers
The empirical emergence of neural collapse -- a surprising symmetry in the feature representations of the training data in the penultimate layer of deep neural networks -- has spurred a line of theoretical research aimed…