Concept Representation Learning with Contrastive Self-Supervised Learning
Concept-oriented deep learning (CODL) is a general approach to meet the future challenges for deep learning: (1) learning with little or no external supervision, (2) coping with test examples that come from a different distribution than the training examples, and (3) integrating deep learning with symbolic AI. In CODL, as in human learning, concept representations are learned based on concept exemplars. Contrastive self-supervised learning (CSSL) provides a promising approach to do so, since it: (1) uses data-driven associations, to get away from semantic labels, (2) supports incremental and continual learning, to get away from (large) fixed datasets, and (3) accommodates emergent objectives, to get away from fixed objectives (tasks). We discuss major aspects of concept representation learning using CSSL. These include dual-level concept representations, CSSL for feature representations, exemplar similarity measures and self-supervised relational reasoning, incremental and continual CSSL, and contrastive self-supervised concept (class) incremental learning. The discussion leverages recent findings from cognitive neural science and CSSL.
Code (0)
등록된 구현이 없습니다.
Tasks
class-incremental learningClass Incremental LearningContinual LearningDeep LearningIncremental LearningRelational ReasoningRepresentation LearningSelf-Supervised LearningSimilar Papers 제목 키워드 기반
Understanding Self-supervised Contrastive Learning through Supervised Objectives
Self-supervised representation learning has achieved impressive empirical success, yet its theoretical understanding remains limited. In this work, we provide a theoretical perspective by formulating self-supervised repr…
Self-Supervised LearningRepresentation LearningContrastive LearningEmbodied-Symbolic Contrastive Graph Self-Supervised Learning for Molecular Graphs
Dual embodied-symbolic concept representations are the foundation for deep learning and symbolic AI integration. We discuss the use of dual embodied-symbolic concept representations for molecular graph representation lea…
Graph Representation LearningRepresentation LearningSelf-Supervised LearningSCD: Self-Contrastive Decorrelation of Sentence Embeddings
In this paper, we propose Self-Contrastive Decorrelation (SCD), a self-supervised approach. Given an input sentence, it optimizes a joint self-contrastive and decorrelation objective. Learning a representation is facilit…
Self-Supervised LearningSentenceSentence EmbeddingsSCD: Self-Contrastive Decorrelation for Sentence Embeddings
In this paper, we propose Self-Contrastive Decorrelation (SCD), a self-supervised approach. Given an input sentence, it optimizes a joint self-contrastive and decorrelation objective. Learning a representation is facilit…
Self-Supervised LearningSentenceSentence EmbeddingsSelf-Distilled Representation Learning for Time Series
Self-supervised learning for time-series data holds potential similar to that recently unleashed in Natural Language Processing and Computer Vision. While most existing works in this area focus on contrastive learning, w…
Contrastive LearningRepresentation LearningSelf-Supervised LearningTime Series