HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection
The High-Resolution Transformer (HRFormer) can maintain high-resolution representation and share global receptive fields. It is friendly towards salient object detection (SOD) in which the input and output have the same resolution. However, two critical problems need to be solved for two-modality SOD. One problem is two-modality fusion. The other problem is the HRFormer output's fusion. To address the first problem, a supplementary modality is injected into the primary modality by using global optimization and an attention mechanism to select and purify the modality at the input level. To solve the second problem, a dual-direction short connection fusion module is used to optimize the output features of HRFormer, thereby enhancing the detailed representation of objects at the output level. The proposed model, named HRTransNet, first introduces an auxiliary stream for feature extraction of supplementary modality. Then, features are injected into the primary modality at the beginning of each multi-resolution branch. Next, HRFormer is applied to achieve forwarding propagation. Finally, all the output features with different resolutions are aggregated by intra-feature and inter-feature interactive transformers. Application of the proposed model results in impressive improvement for driving two-modality SOD tasks, e.g., RGB-D, RGB-T, and light field SOD.https://github.com/liuzywen/HRTransNet
Code (1)
Tasks
global-optimizationObjectobject-detectionObject DetectionSalient Object DetectionVocal Bursts Valence PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
CoLA: Conditional Dropout and Language-driven Robust Dual-modal Salient Object Detection
The depth/thermal information is beneficial for detecting salient object with conventional RGB images. However, in dual-modal salient object detection (SOD) model, the robustness against noisy inputs and modality missing…
CoLALanguage ModelingLanguage Modellingobject-detection+5SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection
Convolutional neural networks (CNNs) are good at extracting contexture features within certain receptive fields, while transformers can model the global long-range dependency features. By absorbing the advantage of trans…
Decoderobject-detectionObject DetectionRGB-T Salient Object Detection+1HRFormer: High-Resolution Transformer for Dense Prediction
We present a High-Resolution Transformer (HRFormer) that learns high-resolution representations for dense prediction tasks, in contrast to the original Vision Transformer that produces low-resolution representations and …
Image ClassificationMulti-Person Pose EstimationPose EstimationPrediction+2HRFormer: High-Resolution Vision Transformer for Dense Predict
We present a High-Resolution Transformer (HRFormer) that learns high-resolution representations for dense prediction tasks, in contrast to the original Vision Transformer that produces low-resolution representations and …
Pose EstimationSemantic SegmentationVocal Bursts Intensity PredictionDEHRFormer: Real-time Transformer for Depth Estimation and Haze Removal from Varicolored Haze Scenes
Varicolored haze caused by chromatic casts poses haze removal and depth estimation challenges. Recent learning-based depth estimation methods are mainly targeted at dehazing first and estimating depth subsequently from h…
Contrastive LearningDepth Estimation