Synesthesia of Machines (SoM)-Based Task-Driven MIMO System for Image Transmission

2025-09-02 · Sijiang Li, Rongqing Zhang, Xiang Cheng, Jian Tang arxiv

To support cooperative perception (CP) of networked mobile agents in dynamic scenarios, the efficient and robust transmission of sensory data is a critical challenge. Deep learning-based joint source-channel coding (JSCC) has demonstrated promising results for image transmission under adverse channel conditions, outperforming traditional rule-based codecs. While recent works have explored to combine JSCC with the widely adopted multiple-input multiple-output (MIMO) technology, these approaches are still limited to the discrete-time analog transmission (DTAT) model and simple tasks. Given the limited performance of existing MIMO JSCC schemes in supporting complex CP tasks for networked mobile agents with digital MIMO communication systems, this paper presents a Synesthesia of Machines (SoM)-based task-driven MIMO system for image transmission, referred to as SoM-MIMO. By leveraging the structural properties of the feature pyramid for perceptual tasks and the channel properties of the closed-loop MIMO communication system, SoM-MIMO enables efficient and robust digital MIMO transmission of images. Experimental results have shown that compared with two JSCC baseline schemes, our approach achieves average mAP improvements of 6.30 and 10.48 across all SNR levels, while maintaining identical communication overhead.

📄 PDF Abstract BibTeX arXiv:2509.02031

Code (0)

등록된 구현이 없습니다.

Synesthesia of Machines (SoM)-Aided FDD Precoding with Sensing Heterogeneity: A Vertical Federated Learning Approach

2025-01-19 · Haotian Zhang, Shijian Gao, Weibo Wen, Xiang Cheng

High complexity in precoding design for frequency division duplex systems necessitates streamlined solutions. Guided by Synesthesia of Machines (SoM), this paper introduces a heterogeneous multi-vehicle, multi-modal sens…

Federated Learning Vertical Federated Learning

Scatterer Recognition for Multi-Modal Intelligent Vehicular Channel Modeling via Synesthesia of Machines

2024-06-27 · Ziwei Huang, Lu Bai, Zengrui Han, Xiang Cheng

In this paper, a novel multi-modal intelligent vehicular channel model is proposed by scatterer recognition from light detection and ranging (LiDAR) point clouds via Synesthesia of Machines (SoM). The proposed model can …

Intelligent Multi-Modal Sensing-Communication Integration: Synesthesia of Machines

2023-06-25 · Xiang Cheng, Haotian Zhang, Jianan Zhang, Shijian Gao 외

In the era of sixth-generation (6G) wireless communications, integrated sensing and communications (ISAC) is recognized as a promising solution to upgrade the physical system by endowing wireless communications with sens…

ISAC

Foundation Model Empowered Synesthesia of Machines (SoM): AI-native Intelligent Multi-Modal Sensing-Communication Integration

2025-06-09 · Xiang Cheng, Boxun Liu, Xuanyu Liu, Ensong Liu 외

To support future intelligent multifunctional sixth-generation (6G) wireless communication networks, Synesthesia of Machines (SoM) is proposed as a novel paradigm for artificial intelligence (AI)-native intelligent multi…

Synesthesia of Machines (SoM)-Enhanced ISAC Precoding for Vehicular Networks with Double Dynamics

2024-08-24 · Zonghui Yang, Shijian Gao, Xiang Cheng, Liuqing Yang

Integrated sensing and communication (ISAC) technology is vital for vehicular networks, yet the time-varying communication channels and rapid movement of targets present significant challenges for real-time precoding des…

Deep Reinforcement Learning Integrated sensing and communication ISAC