CrossOver: 3D Scene Cross-Modal Alignment
Multi-modal 3D object understanding has gained significant attention, yet current approaches often assume complete data availability and rigid alignment across all modalities. We present CrossOver, a novel framework for cross-modal 3D scene understanding via flexible, scene-level modality alignment. Unlike traditional methods that require aligned modality data for every object instance, CrossOver learns a unified, modality-agnostic embedding space for scenes by aligning modalities - RGB images, point clouds, CAD models, floorplans, and text descriptions - with relaxed constraints and without explicit object semantics. Leveraging dimensionality-specific encoders, a multi-stage training pipeline, and emergent cross-modal behaviors, CrossOver supports robust scene retrieval and object localization, even with missing modalities. Evaluations on ScanNet and 3RScan datasets show its superior performance across diverse metrics, highlighting adaptability for real-world applications in 3D scene understanding.
Code (1)
Tasks
cross-modal alignmentObjectObject LocalizationScene UnderstandingSimilar Papers 제목 키워드 기반
Safe Crossover of Neural Networks Through Neuron Alignment
One of the main and largely unexplored challenges in evolving the weights of neural networks using genetic algorithms is to find a sensible crossover operation between parent networks. Indeed, naive crossover leads to fu…
A New Lagrangian Problem Crossover: A Systematic Review and Meta-Analysis of Crossover Standards
The performance of most evolutionary metaheuristic algorithms relays on various operatives. One of them is the crossover operator, which is divided into two types: application dependent and application independent crosso…
Co-Evolutionary Multi-Modal Alignment via Structured Adversarial Evolution
Adversarial behavior plays a central role in aligning large language models with human values. However, existing alignment methods largely rely on static adversarial settings, which fundamentally limit robustness, partic…
SGAligner++: Cross-Modal Language-Aided 3D Scene Graph Alignment
Aligning 3D scene graphs is a crucial initial step for several applications in robot navigation and embodied perception. Current methods in 3D scene graph alignment often rely on single-modality point cloud data and stru…
3D Scene Graph AlignmentVisual LocalizationScene Understanding3D ReconstructionCross-Modal Scene Semantic Alignment for Image Complexity Assessment
Image complexity assessment (ICA) is a challenging task in perceptual evaluation due to the subjective nature of human perception and the inherent semantic diversity in real-world images. Existing ICA methods predominant…