Mean Field Theory in Deep Metric Learning
In this paper, we explore the application of mean field theory, a technique from statistical physics, to deep metric learning and address the high training complexity commonly associated with conventional metric learning loss functions. By adapting mean field theory for deep metric learning, we develop an approach to design classification-based loss functions from pair-based ones, which can be considered complementary to the proxy-based approach. Applying the mean field theory to two pair-based loss functions, we derive two new loss functions, MeanFieldContrastive and MeanFieldClassWiseMultiSimilarity losses, with reduced training complexity. We extensively evaluate these derived loss functions on three image-retrieval datasets and demonstrate that our loss functions outperform baseline methods in two out of the three datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Image RetrievalMetric LearningRetrievalSimilar Papers 제목 키워드 기반
On Bridging the Gap between Mean Field and Finite Width in Deep Random Neural Networks with Batch Normalization
Mean field theory is widely used in the theoretical studies of neural networks. In this paper, we analyze the role of depth in the concentration of mean-field predictions, specifically for deep multilayer perceptron (MLP…
Mean Field Theory for Sigmoid Belief Networks
We develop a mean field theory for sigmoid belief networks based on ideas from statistical mechanics. Our mean field theory provides a tractable approximation to the true probability distribution in these networks; it al…
Deep Mean Field Theory: Layerwise Variance and Width Variation as Methods to Control Gradient Explosion
A recent line of work has studied the statistical properties of neural networks to great success from a {\it mean field theory} perspective, making and verifying very precise predictions of neural network behavior and t…
A Level Set Theory for Neural Implicit Evolution under Explicit Flows
Coordinate-based neural networks parameterizing implicit surfaces have emerged as efficient representations of geometry. They effectively act as parametric level sets with the zero-level set defining the surface of inter…
Inverse RenderingMean field theory for deep dropout networks: digging up gradient backpropagation deeply
In recent years, the mean field theory has been applied to the study of neural networks and has achieved a great deal of success. The theory has been applied to various neural network structures, including CNNs, RNNs, Re…