Expert-guided Regularization via Distance Metric Learning
High-dimensional prediction is a challenging problem setting for traditional statistical models. Although regularization improves model performance in high dimensions, it does not sufficiently leverage knowledge on feature importances held by domain experts. As an alternative to standard regularization techniques, we propose Distance Metric Learning Regularization (DMLreg), an approach for eliciting prior knowledge from domain experts and integrating that knowledge into a regularized linear model. First, we learn a Mahalanobis distance metric between observations from pairwise similarity comparisons provided by an expert. Then, we use the learned distance metric to place prior distributions on coefficients in a linear model. Through experimental results on a simulated high-dimensional prediction problem, we show that DMLreg leads to improvements in model performance when the domain expert is knowledgeable.
Code (0)
등록된 구현이 없습니다.
Tasks
Metric LearningSimilar Papers 제목 키워드 기반
Shape-Guided: Shape-Guided Dual-Memory Learning for 3D Anomaly Detection
We present a shape-guided expert-learning framework to tackle the problem of unsupervised 3D anomaly detection. Our method is established on the effectiveness of two specialized expert models and their synergy to localiz…
3D Anomaly Detection3D Anomaly Detection and SegmentationAnomaly DetectionRGB+3D Anomaly Detection and Segmentation+1Fantope Regularization in Metric Learning
This paper introduces a regularization method to explicitly control the rank of a learned symmetric positive semidefinite distance matrix in distance metric learning. To this end, we propose to incorporate in the objecti…
Face VerificationGeneral Classificationimage-classificationImage Classification+1SMoES: Soft Modality-Guided Expert Specialization in MoE-VLMs
Mixture-of-Experts (MoE) has become a prevalent backbone for large vision-language models (VLMs), yet how modality-specific signals should guide expert routing remains under-explored. Existing routing strategies are eith…
Geometric Regularization in Mixture-of-Experts: The Disconnect Between Weights and Activations
Mixture-of-Experts (MoE) models achieve efficiency through sparse activation, but the role of geometric regularization in expert specialization remains unclear. We apply orthogonality loss to enforce expert diversity and…
HHF: Hashing-guided Hinge Function for Deep Hashing Retrieval
Deep hashing has shown promising performance in large-scale image retrieval. However, latent codes extracted by Deep Neural Networks (DNNs) will inevitably lose semantic information during the binarization process, which…
BinarizationDeep HashingImage RetrievalMetric Learning+2