Large Generative Model Assisted 3D Semantic Communication
Semantic Communication (SC) is a novel paradigm for data transmission in 6G. However, there are several challenges posed when performing SC in 3D scenarios: 1) 3D semantic extraction; 2) Latent semantic redundancy; and 3) Uncertain channel estimation. To address these issues, we propose a Generative AI Model assisted 3D SC (GAM-3DSC) system. Firstly, we introduce a 3D Semantic Extractor (3DSE), which employs generative AI models, including Segment Anything Model (SAM) and Neural Radiance Field (NeRF), to extract key semantics from a 3D scenario based on user requirements. The extracted 3D semantics are represented as multi-perspective images of the goal-oriented 3D object. Then, we present an Adaptive Semantic Compression Model (ASCM) for encoding these multi-perspective images, in which we use a semantic encoder with two output heads to perform semantic encoding and mask redundant semantics in the latent semantic space, respectively. Next, we design a conditional Generative adversarial network and Diffusion model aided-Channel Estimation (GDCE) to estimate and refine the Channel State Information (CSI) of physical channels. Finally, simulation results demonstrate the advantages of the proposed GAM-3DSC system in effectively transmitting the goal-oriented 3D scenario.
Code (0)
등록된 구현이 없습니다.
Tasks
Generative Adversarial NetworkmodelNeRFSemantic CommunicationSemantic CompressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Large Generative Model-assisted Talking-face Semantic Communication System
The rapid development of generative Artificial Intelligence (AI) continually unveils the potential of Semantic Communication (SemCom). However, current talking-face SemCom systems still encounter challenges such as low b…
Language ModelingLanguage ModellingLarge Language ModelSemantic CommunicationCodebook-enabled Generative End-to-end Semantic Communication Powered by Transformer
Codebook-based generative semantic communication attracts increasing attention, since only indices are required to be transmitted when the codebook is shared between transmitter and receiver. However, due to the fact tha…
Image GenerationSemantic CommunicationPersonalized Federated Learning for Generative AI-Assisted Semantic Communications
Semantic Communication (SC) focuses on transmitting only the semantic information rather than the raw data. This approach offers an efficient solution to the issue of spectrum resource utilization caused by the various i…
Federated LearningNetwork PruningPersonalized Federated LearningSemantic CommunicationLow-Rate Semantic Communication with Codebook-based Conditional Generative Models
Generative semantic communication models are reshaping semantic communication frameworks by moving beyond pixel-wise optimization to align with human perception. However, many existing approaches prioritize image-level p…
Saliency DetectionSemantic CommunicationAgent-driven Generative Semantic Communication with Cross-Modality and Prediction
In the era of 6G, with compelling visions of intelligent transportation systems and digital twins, remote surveillance is poised to become a ubiquitous practice. Substantial data volume and frequent updates present chall…
DecoderSemantic CommunicationSemantic Compression