DualDiff: Dual-branch Diffusion Model for Autonomous Driving with Semantic Fusion

2025-05-03 · Haoteng Li, Zhao Yang, Zezhong Qian, Gongpeng Zhao, Yuqi Huang, Jun Yu, Huazheng Zhou, Longjun Liu

Accurate and high-fidelity driving scene reconstruction relies on fully leveraging scene information as conditioning. However, existing approaches, which primarily use 3D bounding boxes and binary maps for foreground and background control, fall short in capturing the complexity of the scene and integrating multi-modal information. In this paper, we propose DualDiff, a dual-branch conditional diffusion model designed to enhance multi-view driving scene generation. We introduce Occupancy Ray Sampling (ORS), a semantic-rich 3D representation, alongside numerical driving scene representation, for comprehensive foreground and background control. To improve cross-modal information integration, we propose a Semantic Fusion Attention (SFA) mechanism that aligns and fuses features across modalities. Furthermore, we design a foreground-aware masked (FGM) loss to enhance the generation of tiny objects. DualDiff achieves state-of-the-art performance in FID score, as well as consistently better results in downstream BEV segmentation and 3D object detection tasks.

📄 PDF Abstract BibTeX arXiv:2505.01857

Code (1)

yangzhaojason/DualDiff pytorch

Tasks

3D Object Detection Autonomous Driving BEV Segmentation object-detection Object Detection Scene Generation

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

DualDiff+: Dual-Branch Diffusion for High-Fidelity Video Generation with Reward Guidance

2025-03-05 · Zhao Yang, Zezhong Qian, Xiaofan Li, Weixiang Xu 외

Accurate and high-fidelity driving scene reconstruction demands the effective utilization of comprehensive scene information as conditional inputs. Existing methods predominantly rely on 3D bounding boxes and BEV road ma…

3D Object Detection BEV Segmentation Denoising Image to Video Generation+2

DualDiffusion: A Speculative Decoding Strategy for Masked Diffusion Models

2026-04-06 · Satyam Goyal, Kushal Patel, Tanush Mittal, Arjun Laxman arxiv

Masked Diffusion Models (MDMs) offer a promising alternative to autoregressive language models by enabling parallel token generation and bidirectional context modeling. However, their inference speed is significantly lim…

GradualDiff-Fed: A Federated Learning Specialized Framework for Large Language Model

2025-06-23 · Amir Faiyaz, Tara Salman

The rapid proliferation of large language models (LLMs) has created an unprecedented demand for fine-tuning models for specialized domains, such as medical science. While federated learning (FL) offers a decentralized an…

Federated Learning Language Modeling Language Modelling Large Language Model+1

Driving in Spikes: An Entropy-Guided Object Detector for Spike Cameras

2025-11-19 · Ziyan Liu, Qi Su, Lulu Tang, Zhaofei Yu 외 arxiv

Object detection in autonomous driving suffers from motion blur and saturation under fast motion and extreme lighting. Spike cameras, offer microsecond latency and ultra high dynamic range for object detection by using p…

Autonomous Driving Object Detection

Autonomous Driving using Residual Sensor Fusion and Deep Reinforcement Learning

2023-12-27 · Amin Jalal Aghdasian, Amirhossein Heydarian Ardakani, Kianoush Aqabakee, Farzaneh Abdollahi

This paper proposes a novel approach by integrating sensor fusion with deep reinforcement learning, specifically the Soft Actor-Critic (SAC) algorithm, to develop an optimal control policy for self-driving cars. Our syst…

Autonomous Driving Decision Making Deep Reinforcement Learning reinforcement-learning+3