Papers Cross-Modal Retrieval
“Cross-Modal Retrieval” 태그가 달린 논문 677편 · 필터 해제
AVSD-Scenes: A Dataset for Audio-Visual Description of Urban Scenes
Natural language descriptions can provide rich semantic representations of audio-visual urban scenes, yet datasets that jointly describe both auditory and visual information remain limited. In this paper, we introduce AV…
Cross-Modal RetrievalScene ClassificationHub-Spectral Activation of Latent Multimodal Knowledge
Multimodal representation learning seeks shared representations for cross-modal retrieval and knowledge transfer. Hub-based binding reduces pairwise supervision costs, but separate hub connections cannot guarantee reliab…
Representation LearningCross-Modal RetrievalFLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation
Traditional multimodal representation learning and generation are two stages: a contrastive or self-supervised visual encoder is trained first, followed by a separate downstream generative model. This setup bottlenecks g…
Representation LearningCross-Modal RetrievalImage CaptioningAbstract4D: A Large-Scale Dataset and Framework for Understanding the Visual Language of Abstract Art
Artificial intelligence can classify artistic styles and synthesize images, but it still lacks a model of the visual language that gives art meaning. Abstract painting minimizes object semantics and foregrounds structura…
Text-to-Image GenerationCross-Modal RetrievalAutoResearch: Insight In, Hallucination Out
Autonomous research systems are increasingly capable of executing long research workflows, yet automation alone does not ensure that the resulting process remains scientifically grounded. We introduce AutoResearch, a two…
Cross-Modal RetrievalSCALPEL: Semantic Cross-modal Alignment via LLM-Powered Encoder Learning for Medical Vision-Language Representation
Vision-language pre-training (VLP) serves as a cornerstone for medical multimodal representation learning. However, existing medical VLP frameworks are often constrained by the limited context windows and shallow represe…
Visual Question AnsweringRepresentation LearningCross-Modal RetrievalNot All Patches are Equal: Sampling Matters for Visible-Infrared Pre-Training
Visible-infrared (VIS-IR) alignment is a key pre-training task for robust multi-sensor perception. Most existing methods use uniform patch-wise contrastive learning, but this can be unreliable in VIS-IR data because imag…
Representation LearningCross-Modal RetrievalSemantic SegmentationContrastive LearningAlphaWiSE: Adaptive Weight Interpolation for Continual Multimodal Representation Learning
Multimodal models such as CLIP learn a shared embedding space for cross-modal retrieval, but continual adaptation to sequentially arriving data can disrupt the cross-modal alignment acquired from earlier phases. Conventi…
Representation LearningCross-Modal RetrievalText RetrievalAttribute-Prompted Kernel Hashing for Unsupervised Data-Efficient Cross-Modal Retrieval
Unsupervised cross-modal hashing enables efficient retrieval of semantically related instances across different modalities without requiring manual semantic annotation. However, existing unsupervised methods rely heavily…
Cross-Modal RetrievalContrastive LearningHyFL-CLIP: Hyperbolic Fine-Tuning of CLIP for Robust Long-Context Understanding
CLIP (Contrastive Language-Image Pre-training) has become a de facto paradigm for image-text alignment, but it struggles with long-context descriptions (>77 tokens) due to absolute positional encoding and pretraining on …
Long-Context UnderstandingCross-Modal RetrievalUnsupervised Data-Efficient Cross-Modal Retrieval with Global-Neighborhood Alignment Hashing
Compared to supervised cross-modal hashing (CMH), unsupervised CMH reduces the reliance on manual labeling by learning binary codes from unlabeled image-text pairs. However, existing unsupervised CMH methods often rely o…
Cross-Modal RetrievalContrastive LearningSame Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders
Vision-language models map images and text into a joint embedding space. However, these embeddings often entangle multiple semantic features, which limits their interpretability and controllability. While sparse autoenco…
Cross-Modal RetrievalRank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation
Vision-language dataset distillation (VLDD) compresses a large image-text paired dataset into a small set of synthetic pairs that can efficiently train contrastive vision-language models under strict data and compute bud…
Cross-Modal RetrievalSee & Sniff: Learning Visuo-Olfactory Representations
While modern multimodal models integrate vision with language, audio, or touch, olfaction remains largely unexplored due to the lack of paired visuo-olfactory data. We introduce SmellNet-V, a scalable visuo-olfactory dat…
Cross-Modal RetrievalVideoAgent: All-in-One Framework for Video Understanding and Editing
Video editing has become essential in digital media creation, yet existing automated systems are restricted to short segment processing and domain-specific tasks. They face two critical limitations: i) inability to handl…
Cross-Modal RetrievalSARLO-80: Worldwide Slant SAR Language Optic Dataset 80cm
Multimodal foundation models have advanced rapidly thanks to large optical benchmarks, but comparable resources for synthetic aperture radar (SAR) remain limited. Existing SAR--optical datasets largely rely on low-resolu…
Cross-Modal RetrievalDREAM: Extending Vision-Language Models with Dual-Objective Encoding for Cross-Modal Retrieval
In today's media-driven world, the exponential growth of video content across domains such as surveillance, education, and entertainment has made retrieving semantically relevant videos via natural language queries incre…
Natural Language QueriesRepresentation LearningCross-Modal RetrievalVideo RetrievalMechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Heads
While Multimodal Large Language Models (MLLMs) demonstrate remarkable proficiency on complex vision-language tasks, the mechanisms by which they extract query-relevant visual features from complex, noisy contexts remain …
Cross-Modal RetrievalMultimodal ReasoningIntra-Modal Neighbors Never Lie: Rectifying Inter-Modal Noisy Correspondence via Graph-Based Intra-Modal Reasoning
Large-scale web-harvested datasets have fueled the progress of cross-modal retrieval but inevitably suffer from noisy correspondence, which severely degrades model generalization. Existing methods primarily address this …
Cross-Modal RetrievalRelational ReasoningTowards Resolving Optimization Conflicts Between Image- and Text-Based Person Re-Identification
The joint optimization of image-based (I2I) and text-based (T2I) person re-identification (ReID) is hindered by modality discrepancies and conflicting training objectives, leading to suboptimal shared representations. Wh…
Person Re-IdentificationCross-Modal Retrieval