Information distance for neural network functions
We provide a practical distance measure in the space of functions parameterized by neural networks. It is based on classic information distance and we replace the uncomputable Kolmogorov complexity in the original definition with information measured by codelength of prequential coding. Empirically, we show that information distance is invariant with respect to different parameterization of the functions with neural networks. We also verify that information distance can faithfully reflect similarities of neural network functions. Finally, we applied information distance to investigate the relationship between neural network models, and demonstrate the connection between information distance and multiple characteristics and behaviors of neural network models.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
B-Rep Distance Functions (BR-DF): How to Represent a B-Rep Model by Volumetric Distance Functions?
This paper presents a novel geometric representation for CAD Boundary Representation (B-Rep) based on volumetric distance functions, dubbed B-Rep Distance Functions (BR-DF). BR-DF encodes the surface mesh geometry of a C…
Learning Bregman Distance Functions and Its Application for Semi-Supervised Clustering
Learning distance functions with side information plays a key role in many machine learning and data mining applications. Conventional approaches often assume a Mahalanobis distance function. These approaches are limited…
ClusteringA new class of metrics for learning on real-valued and structured data
We propose a new class of metrics on sets, vectors, and functions that can be used in various stages of data mining, including exploratory data analysis, learning, and result interpretation. These new distance functions …
Loss it right: Euclidean and Riemannian Metrics in Learning-based Visual Odometry
This paper overviews different pose representations and metric functions in visual odometry (VO) networks. The performance of VO networks heavily relies on how their architecture encodes the information. The choice of po…
Motion EstimationVisual OdometryPoint-to-set distance functions for weakly supervised segmentation
When pixel-level masks or partial annotations are not available for training neural networks for semantic segmentation, it is possible to use higher-level information in the form of bounding boxes, or image tags. In the …
ObjectSegmentationSemantic SegmentationWeakly supervised segmentation