Papers Unsupervised Pre-training
“Unsupervised Pre-training” 태그가 달린 논문 277편 · 필터 해제
Unsupervised Multidomain Approaches to Named Entity Recognition with Small Datasets
This paper explores the challenges and the methodologies associated with learning quality representations in scenarios with unlabelled small or limited datasets for downstream information extraction task (Multidomain Nam…
Unsupervised Pre-trainingInformation ExtractionData AugmentationTransfer LearningFrom Pixels to Temporal Correlations: Learning Informative Representations for Reinforcement Learning Pre-training
Unsupervised pre-training on large-scale datasets has demonstrated significant potential for improving the sample efficiency and performance of Reinforcement Learning (RL). Given the large-scale action-free internet vide…
Unsupervised Pre-trainingReinforcement LearningImage ReconstructionContrastive LearningPolarMAE: Efficient Fetal Ultrasound Pre-training via Semantic Screening and Polar-Guided Masking
Intelligent fetal ultrasound (US) interpretation is crucial for prenatal diagnosis, but high annotation costs and operator-induced variance make unsupervised pre-training a highly promising paradigm. However, existing pr…
Unsupervised Pre-trainingAttribution as Retrieval: Model-Agnostic AI-Generated Image Attribution
With the rapid advancement of AIGC technologies, image forensics will encounter unprecedented challenges. Traditional methods are incapable of dealing with increasingly realistic images generated by rapidly evolving imag…
Unsupervised Pre-trainingImage ClassificationDeepFake DetectionImage AttributionUnsupervised Learning of Efficient Exploration: Pre-training Adaptive Policies via Self-Imposed Goals
Unsupervised pre-training can equip reinforcement learning agents with prior knowledge and accelerate learning in downstream tasks. A promising direction, grounded in human development, investigates agents that learn by …
Unsupervised Pre-trainingReinforcement LearningCommunity Norms in the Spotlight: Enabling Task-Agnostic Unsupervised Pre-Training to Benefit Online Social Media
Modelling the complex dynamics of online social platforms is critical for addressing challenges such as hate speech and misinformation. While Discussion Transformers, which model conversations as graph structures, have e…
Unsupervised Pre-trainingCLIP-Guided Adaptable Self-Supervised Learning for Human-Centric Visual Tasks
Human-centric visual analysis plays a pivotal role in diverse applications, including surveillance, healthcare, and human-computer interaction. With the emergence of large-scale unlabeled human image datasets, there is a…
Unsupervised Pre-trainingSelf-Supervised LearningRepresentation LearningSEMDICE: Off-policy State Entropy Maximization via Stationary Distribution Correction Estimation
In the unsupervised pre-training for reinforcement learning, the agent aims to learn a prior policy for downstream tasks without relying on task-specific reward functions. We focus on state entropy maximization (SEM), wh…
Unsupervised Pre-trainingReinforcement LearningGraph Attention Network for Predicting Duration of Large-Scale Power Outages Induced by Natural Disasters
Natural disasters such as hurricanes, wildfires, and winter storms have induced large-scale power outages in the U.S., resulting in tremendous economic and societal impacts. Accurately predicting power outage recovery an…
Unsupervised Pre-trainingMasked Clustering Prediction for Unsupervised Point Cloud Pre-training
Vision transformers (ViTs) have recently been widely applied to 3D point cloud understanding, with masked autoencoding as the predominant pre-training paradigm. However, the challenge of learning dense and informative se…
Unsupervised Pre-trainingSemantic SegmentationContrastive LearningObject DetectionGeneralized few-shot transfer learning architecture for modeling the EDFA gain spectrum
Accurate modeling of the gain spectrum in Erbium-Doped Fiber Amplifiers (EDFAs) is essential for optimizing optical network performance, particularly as networks evolve toward multi-vendor solutions. In this work, we pro…
Unsupervised Pre-trainingTransfer LearningFocus on Texture: Rethinking Pre-training in Masked Autoencoders for Medical Image Classification
Masked Autoencoders (MAEs) have emerged as a dominant strategy for self-supervised representation learning in natural images, where models are pre-trained to reconstruct masked patches with a pixel-wise mean squared erro…
Gallbladder Cancer DetectionMedical Image ClassificationUnsupervised Pre-trainingRepresentation LearningQwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
In this work, we introduce the Qwen3 Embedding series, a significant advancement over its predecessor, the GTE-Qwen series, in text embedding and reranking capabilities, built upon the Qwen3 foundation models. Leveraging…
RerankingRetrievalUnsupervised Pre-trainingSPACE: Your Genomic Profile Predictor is a Powerful DNA Foundation Model
Inspired by the success of unsupervised pre-training paradigms, researchers have applied these approaches to DNA pre-training. However, we argue that these approaches alone yield suboptimal results because pure DNA seque…
Mixture-of-ExpertsUnsupervised Pre-trainingFoundation Model for Wireless Technology Recognition Using IQ Timeseries
Wireless Technology Recognition (WTR) is essential in modern communication systems, enabling efficient spectrum management and the seamless coexistence of diverse technologies. In real-world conditions, WTR solutions sho…
Computational EfficiencyUnsupervised Pre-trainingIs MixIT Really Unsuitable for Correlated Sources? Exploring MixIT for Unsupervised Pre-training in Music Source Separation
In music source separation (MSS), obtaining isolated sources or stems is highly costly, making pre-training on unlabeled data a promising approach. Although source-agnostic unsupervised learning like mixture-invariant tr…
Music Source SeparationUnsupervised Pre-trainingThe Efficiency of Pre-training with Objective Masking in Pseudo Labeling for Semi-Supervised Text Classification
We extend and study a semi-supervised model for text classification proposed earlier by Hatefi et al. for classification tasks in which document classes are described by a small number of gold-labeled examples, while the…
Semi-Supervised Text Classificationtext-classificationText ClassificationUnsupervised Pre-trainingLatte: Transfering LLMs` Latent-level Knowledge for Few-shot Tabular Learning
Few-shot tabular learning, in which machine learning models are trained with a limited amount of labeled data, provides a cost-effective approach to addressing real-world challenges. The advent of Large Language Models (…
Feature EngineeringGeneral KnowledgeUnsupervised Pre-trainingRisk Assessment Framework for Code LLMs via Leveraging Internal States
The pre-training paradigm plays a key role in the success of Large Language Models (LLMs), which have been recognized as one of the most significant advancements of AI recently. Building on these breakthroughs, code LLMs…
Unsupervised Pre-trainingZS-VCOS: Zero-Shot Outperforms Supervised Video Camouflaged Object Segmentation
Camouflaged object segmentation presents unique challenges compared to traditional segmentation tasks, primarily due to the high similarity in patterns and colors between camouflaged objects and their backgrounds. Effect…
Camouflaged Object SegmentationDefect DetectionLesion SegmentationOptical Flow Estimation+3