Contrastive Self-Supervised Learning at the Edge: An Energy Perspective
While contrastive learning (CL) shows considerable promise in self-supervised representation learning, its deployment on resource-constrained devices remains largely underexplored. The substantial computational demands required for training conventional CL frameworks pose a set of challenges, particularly in terms of energy consumption, data availability, and memory usage. We conduct an evaluation of four widely used CL frameworks: SimCLR, MoCo, SimSiam, and Barlow Twins. We focus on the practical feasibility of these CL frameworks for edge and fog deployment, and introduce a systematic benchmarking strategy that includes energy profiling and reduced training data conditions. Our findings reveal that SimCLR, contrary to its perceived computational cost, demonstrates the lowest energy consumption across various data regimes. Finally, we also extend our analysis by evaluating lightweight neural architectures when paired with CL frameworks. Our study aims to provide insights into the resource implications of deploying CL in edge/fog environments with limited processing capabilities and opens several research directions for its future optimization.
Code (0)
등록된 구현이 없습니다.
Tasks
Self-Supervised LearningRepresentation LearningContrastive LearningSimilar Papers 제목 키워드 기반
A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives
Modern natural language processing (NLP) methods employ self-supervised pretraining objectives such as masked language modeling to boost the performance of various application tasks. These pretraining methods are frequen…
Contrastive LearningLanguage ModelingLanguage ModellingMasked Language Modeling+3Representation Learning via Consistent Assignment of Views to Clusters
We introduce Consistent Assignment for Representation Learning (CARL), an unsupervised learning method to learn visual representations by combining ideas from self-supervised contrastive learning and deep clustering. By …
ClusteringContrastive LearningDeep ClusteringLinear evaluation+2Self-Supervised Learning at the Edge: The Cost of Labeling
Contrastive learning (CL) has recently emerged as an alternative to traditional supervised machine learning solutions by enabling rich representations from unstructured and unlabeled data. However, CL and, more broadly, …
Self-Supervised LearningContrastive LearningRefining Self-Supervised Learning in Imaging: Beyond Linear Metric
We introduce in this paper a new statistical perspective, exploiting the Jaccard similarity metric, as a measure-based metric to effectively invoke non-linear features in the loss of self-supervised contrastive learning.…
Contrastive LearningSelf-Supervised LearningFusion Self-supervised Learning for Recommendation
Recommender systems are widely deployed in various web environments, and self-supervised learning (SSL) has recently attracted significant attention in this field. Contrastive learning (CL) stands out as a major SSL para…
Contrastive LearningData AugmentationRecommendation SystemsSelf-Supervised Learning