Emergence of Latent Binary Encoding in Deep Neural Network Classifiers
We investigate the emergence of binary encoding within the latent space of deep-neural-network classifiers. Such binary encoding is induced by the introduction of a linear penultimate layer, which employs during training a loss function specifically designed to compress the latent representations. As a result of a trade-off between compression and information retention, the network learns to assume only one of two possible values for each dimension in the latent space. The binary encoding is provoked by the collapse of all representations of the same class to the same point, which corresponds to the vertex of a hypercube. By analyzing several datasets of increasing complexity, we provide empirical evidence that the emergence of binary encoding dramatically enhances robustness while also significantly improving the reliability and generalization of the network.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Adiabatic Encoding of Pre-trained MPS Classifiers into Quantum Circuits
Although Quantum Neural Networks (QNNs) offer powerful methods for classification tasks, the training of QNNs faces two major training obstacles: barren plateaus and local minima. A promising solution is to first train a…
Scaling Laws in Jet Classification
We demonstrate the emergence of scaling laws in the benchmark top versus QCD jet classification problem in collider physics. Six distinct physically-motivated classifiers exhibit power-law scaling of the binary cross-ent…
ClassificationEffectiveness of Binary Autoencoders for QUBO-Based Optimization Problems
In black-box combinatorial optimization, objective evaluations are often expensive, so high quality solutions must be found under a limited budget. Factorization machine with quantum annealing (FMQA) builds a quadratic s…
SCONE: A Practical, Constraint-Aware Plug-in for Latent Encoding in Learned DNA Storage
DNA storage has matured from concept to practical stage, yet its integration with neural compression pipelines remains inefficient. Early DNA encoders applied redundancy-heavy constraint layers atop raw binary data - wor…
Label Encoding for Regression Networks
Deep neural networks are used for a wide range of regression problems. However, there exists a significant gap in accuracy between specialized approaches and generic direct regression in which a network is trained by min…
Binary Classificationregression