Task-Oriented Multi-User Semantic Communications for VQA Task
Semantic communications focus on the transmission of semantic features. In this letter, we consider a task-oriented multi-user semantic communication system for multimodal data transmission. Particularly, partial users transmit images while the others transmit texts to inquiry the information about the images. To exploit the correlation among the multimodal data from multiple users, we propose a deep neural network enabled semantic communication system, named MU-DeepSC, to execute the visual question answering (VQA) task as an example. Specifically, the transceiver for MU-DeepSC is designed and optimized jointly to capture the features from the correlated multimodal data for task-oriented transmission. Simulation results demonstrate that the proposed MU-DeepSC is more robust to channel variations than the traditional communication systems, especially in the low signal-to-noise (SNR) regime.
Code (1)
Tasks
Question AnsweringSemantic CommunicationVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
Federated Latent Space Alignment for Multi-user Semantic Communications
Semantic communication aims to convey meaning for effective task execution, but differing latent representations in AI-native devices can cause semantic mismatches that hinder mutual understanding. This paper introduces …
Semantic CommunicationPersonalized Saliency in Task-Oriented Semantic Communications: Image Transmission and Performance Analysis
Semantic communication, as a promising technology, has emerged to break through the Shannon limit, which is envisioned as the key enabler and fundamental paradigm for future 6G networks and applications, e.g., smart heal…
Image RetrievalRetrievalSemantic CommunicationTask-Oriented Multi-User Semantic Communications
While semantic communications have shown the potential in the case of single-modal single-users, its applications to the multi-user scenario remain limited. In this paper, we investigate deep learning (DL) based multi-us…
Image RetrievalMachine TranslationQuestion AnsweringRetrieval+4Will 6G be Semantic Communications? Opportunities and Challenges from Task Oriented and Secure Communications to Integrated Sensing
This paper explores opportunities and challenges of task (goal)-oriented and semantic communications for next-generation (NextG) communication networks through the integration of multi-task learning. This approach employ…
Federated LearningMulti-Task LearningRobust DesignMultimodal LLM Integrated Semantic Communications for 6G Immersive Experiences
6G networks promise revolutionary immersive communication experiences including augmented reality (AR), virtual reality (VR), and holographic communications. These applications demand high-dimensional multimodal data tra…
Visual Question AnsweringImage Generation