Barcodes as Summary of Loss Function Topology
We propose to study neural networks' loss surfaces by methods of topological data analysis. We suggest to apply barcodes of Morse complexes to explore topology of loss surfaces. An algorithm for calculations of the loss function's barcodes of local minima is described. We have conducted experiments for calculating barcodes of local minima for benchmark functions and for loss surfaces of small neural networks. Our experiments confirm our two principal observations for neural networks' loss surfaces. First, the barcodes of local minima are located in a small lower part of the range of values of neural networks' loss function. Secondly, increase of the neural network's depth and width lowers the barcodes of local minima. This has some natural implications for the neural network's learning and for its generalization properties.
Code (1)
Tasks
Topological Data AnalysisSimilar Papers 제목 키워드 기반
Barcodes as summary of objective functions' topology
We apply canonical forms of gradient complexes (barcodes) to explore neural networks loss surfaces. We present an algorithm for calculations of the objective function's barcodes of minima. Our experiments confirm two pr…
Topologically faithful image segmentation via induced matching of persistence barcodes
Image segmentation is a largely researched field where neural networks find vast applications in many facets of technology. Some of the most popular approaches to train segmentation networks employ loss functions optimiz…
Image SegmentationSegmentationSemantic SegmentationScalar Function Topology Divergence: Comparing Topology of 3D Objects
We propose a new topological tool for computer vision - Scalar Function Topology Divergence (SFTD), which measures the dissimilarity of multi-scale topology between sublevel sets of two functions having a common domain. …
Efficient Betti Matching Enables Topology-Aware 3D Segmentation via Persistent Homology
In this work, we propose an efficient algorithm for the calculation of the Betti matching, which can be used as a loss function to train topology aware segmentation networks. Betti matching loss builds on techniques from…
SegmentationTopological Data AnalysisTopological Classification in a Wasserstein Distance Based Vector Space
Classification of large and dense networks based on topology is very difficult due to the computational challenges of extracting meaningful topological features from real-world networks. In this paper we present a comput…
Classification