Many Heads but One Brain: Fusion Brain -- a Competition and a Single Multimodal Multitask Architecture
Supporting the current trend in the AI community, we present the AI Journey 2021 Challenge called Fusion Brain, the first competition which is targeted to make the universal architecture which could process different modalities (in this case, images, texts, and code) and solve multiple tasks for vision and language. The Fusion Brain Challenge combines the following specific tasks: Code2code Translation, Handwritten Text recognition, Zero-shot Object Detection, and Visual Question Answering. We have created datasets for each task to test the participants' submissions on it. Moreover, we have collected and made publicly available a new handwritten dataset in both English and Russian, which consists of 94,128 pairs of images and texts. We also propose a multimodal and multitask architecture - a baseline solution, in the center of which is a frozen foundation model and which has been trained in Fusion mode along with Single-task mode. The proposed Fusion approach proves to be competitive and more energy-efficient compared to the task-specific one.
Code (1)
Tasks
Handwritten Text Recognitionobject-detectionObject DetectionQuestion AnsweringTranslationVisual Question AnsweringVisual Question Answering (VQA)Zero-Shot Object DetectionSimilar Papers 제목 키워드 기반
Secrets of the Brain: An Introduction to the Brain Anatomical Structure and Biological Function
In this paper, we will provide an introduction to the brain structure and function. Brain is an astonishing living organ inside our heads, weighing about 1.5kg, consisting of billions of tiny cells. The brain enables us …
ArticlesAn Investigation on Non-Invasive Brain-Computer Interfaces: Emotiv Epoc+ Neuroheadset and Its Effectiveness
In this study, we illustrate the progress of BCI research and present scores of unveiled contemporary approaches. First, we explore a decoding natural speech approach that is designed to decode human speech directly from…
EEGElectroencephalogram (EEG)SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense
While vertical thinking relies on logical and commonsense reasoning, lateral thinking requires systems to defy commonsense associations and overwrite them through unconventional thinking. Lateral thinking has been shown …
Common Sense ReasoningMulti-Head Graph Convolutional Network for Structural Connectome Classification
We tackle classification based on brain connectivity derived from diffusion magnetic resonance images. We propose a machine-learning model inspired by graph convolutional networks (GCNs), which takes a brain connectivity…
Classification3D U-Net Based Brain Tumor Segmentation and Survival Days Prediction
Past few years have witnessed the prevalence of deep learning in many application scenarios, among which is medical image processing. Diagnosis and treatment of brain tumors requires an accurate and reliable segmentation…
Brain Tumor SegmentationTumor Segmentation