A Semantic Loss Function for Deep Learning with Symbolic Knowledge
This paper develops a novel methodology for using symbolic knowledge in deep learning. From first principles, we derive a semantic loss function that bridges between neural output vectors and logical constraints. This loss function captures how close the neural network is to satisfying the constraints on its output. An experimental evaluation shows that it effectively guides the learner to achieve (near-)state-of-the-art results on semi-supervised multi-class classification. Moreover, it significantly increases the ability of the neural network to predict structured objects, such as rankings and paths. These discrete concepts are tremendously difficult to learn, and benefit from a tight integration of deep learning and symbolic reasoning methods.
Code (1)
Tasks
Deep LearningGeneral ClassificationMulti-class ClassificationSimilar Papers 제목 키워드 기반
Neuro-Symbolic Approaches for Context-Aware Human Activity Recognition
Deep Learning models are a standard solution for sensor-based Human Activity Recognition (HAR), but their deployment is often limited by labeled data scarcity and models' opacity. Neuro-Symbolic AI (NeSy) provides an int…
Activity RecognitionHuman Activity RecognitionZero-Shot Learning with Knowledge Enhanced Visual Semantic Embeddings
We improve zero-shot learning (ZSL) by incorporating common-sense knowledge in DNNs. We propose Common-Sense based Neuro-Symbolic Loss (CSNL) that formulates prior knowledge as novel neuro-symbolic loss functions that re…
Common Sense ReasoningZero-Shot LearningSemantic Loss Application to Entity Relation Recognition
Usually, entity relation recognition systems either use a pipe-lined model that treats the entity tagging and relation identification as separate tasks or a joint model that simultaneously identifies the relation and ent…
Feature EngineeringNatural Language UnderstandingRelationRelation ExtractionT-Norms Driven Loss Functions for Machine Learning
Neural-symbolic approaches have recently gained popularity to inject prior knowledge into a learner without requiring it to induce this knowledge from data. These approaches can potentially learn competitive solutions wi…
BIG-bench Machine LearningGeneral KnowledgeSemantic Loss Functions for Neuro-Symbolic Structured Prediction
Structured output prediction problems are ubiquitous in machine learning. The prominent approach leverages neural networks as powerful feature extractors, otherwise assuming the independence of the outputs. These outputs…
PredictionStructured Prediction