Perspectives and Prospects on Transformer Architecture for Cross-Modal Tasks with Language and Vision
Transformer architectures have brought about fundamental changes to computational linguistic field, which had been dominated by recurrent neural networks for many years. Its success also implies drastic changes in cross-modal tasks with language and vision, and many researchers have already tackled the issue. In this paper, we review some of the most critical milestones in the field, as well as overall trends on how transformer architecture has been incorporated into visuolinguistic cross-modal tasks. Furthermore, we discuss its current limitations and speculate upon some of the prospects that we find imminent.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Towards Multi-modal Transformers in Federated Learning
Multi-modal transformers mark significant progress in different domains, but siloed high-quality data hinders their further improvement. To remedy this, federated learning (FL) has emerged as a promising privacy-preservi…
Federated LearningPrivacy PreservingLongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecture
Expanding the long-context capabilities of Multi-modal Large Language Models~(MLLMs) is crucial for video understanding, high-resolution image understanding, and multi-modal agents. This involves a series of systematic o…
GPUMambaVideo UnderstandingA Survey on Transformers in Reinforcement Learning
Transformer has been considered the dominating neural architecture in NLP and CV, mostly under supervised settings. Recently, a similar surge of using Transformers has appeared in the domain of reinforcement learning (RL…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)SurveyGeneralist Multimodal AI: A Review of Architectures, Challenges and Opportunities
Multimodal models are expected to be a critical component to future advances in artificial intelligence. This field is starting to grow rapidly with a surge of new design elements motivated by the success of foundation m…
Enhancing CT Image synthesis from multi-modal MRI data based on a multi-task neural network framework
Image segmentation, real-value prediction, and cross-modal translation are critical challenges in medical imaging. In this study, we propose a versatile multi-task neural network framework, based on an enhanced Transform…
Image GenerationImage SegmentationSegmentationSemantic Segmentation+1