T-Norms Driven Loss Functions for Machine Learning
Neural-symbolic approaches have recently gained popularity to inject prior knowledge into a learner without requiring it to induce this knowledge from data. These approaches can potentially learn competitive solutions with a significant reduction of the amount of supervised data. A large class of neural-symbolic approaches is based on First-Order Logic to represent prior knowledge, relaxed to a differentiable form using fuzzy logic. This paper shows that the loss function expressing these neural-symbolic learning tasks can be unambiguously determined given the selection of a t-norm generator. When restricted to supervised learning, the presented theoretical apparatus provides a clean justification to the popular cross-entropy loss, which has been shown to provide faster convergence and to reduce the vanishing gradient problem in very deep structures. However, the proposed learning formulation extends the advantages of the cross-entropy loss to the general knowledge that can be represented by a neural-symbolic method. Therefore, the methodology allows the development of a novel class of loss functions, which are shown in the experimental results to lead to faster convergence rates than the approaches previously proposed in the literature.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningGeneral KnowledgeSimilar Papers 제목 키워드 기반
Real order total variation with applications to the loss functions in learning schemes
Loss function are an essential part in modern data-driven approach, such as bi-level training scheme and machine learnings. In this paper we propose a loss function consisting of a $r$-order (an)-isotropic total variatio…
On the relation between Loss Functions and T-Norms
Deep learning has been shown to achieve impressive results in several domains like computer vision and natural language processing. A key element of this success has been the development of new loss functions, like the p…
RelationSocial Norms-Grounded Machine Ethics in Complex Narrative Situation
Ethical judgment aims to determine if a person in a narrative situation acts under people’s social norms under a culture, so it is crucial to understand actions in narratives and achieve machine ethics. Recent works depe…
Cultural Vocal Bursts Intensity PredictionEthicsTraining Universal Adversarial Perturbations with Alternating Loss Functions
Despite being very successful, deep learning models were shown to be vulnerable to crafted perturbations. Furthermore, changing the prediction of a network over any image by learning a single universal adversarial pertur…
Generalization Bounds and Model Complexity for Kolmogorov-Arnold Networks
Kolmogorov-Arnold Network (KAN) is a network structure recently proposed by Liu et al. (2024) that offers improved interpretability and a more parsimonious design in many science-oriented tasks compared to multi-layer pe…
Generalization BoundsKolmogorov-Arnold Networks