Pushing the Limits of Radiology with Joint Modeling of Visual and Textual Information
Recently, there has been increasing interest in the intersection of computer vision and natural language processing. Researchers have studied several interesting tasks, including generating text descriptions from images and videos and language embedding of images. More recent work has further extended the scope of this area to combine videos and language, learning to solve non-visual tasks using visual cues, visual question answering, and visual dialog. Despite a large body of research on the intersection of vision-language technology, its adaption to the medical domain is not fully explored. To address this research gap, we aim to develop machine learning models that can reason jointly on medical images and clinical text for advanced search, retrieval, annotation and description of medical images.
Code (0)
등록된 구현이 없습니다.
Tasks
Image ClassificationMachine TranslationObject DetectionQuestion AnsweringRetrievalSemantic SegmentationVisual DialogVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
Joint Modeling of Chest Radiographs and Radiology Reports for Pulmonary Edema Assessment
We propose and demonstrate a novel machine learning algorithm that assesses pulmonary edema severity from chest radiographs. While large publicly available datasets of chest radiographs and free-text radiology reports ex…
image-classificationImage ClassificationRepresentation LearningJPG - Jointly Learn to Align: Automated Disease Prediction and Radiology Report Generation
Automated radiology report generation aims to generate paragraphs that describe fine-grained visual differences among cases, especially those between the normal and the diseased. Existing methods seldom consider the cros…
cross-modal alignmentDisease PredictionImage CaptioningMedical Report Generation+1HybVIO: Pushing the Limits of Real-time Visual-inertial Odometry
We present HybVIO, a novel hybrid approach for combining filtering-based visual-inertial odometry (VIO) with optimization-based SLAM. The core of our method is highly robust, independent VIO with improved IMU bias modeli…
Free Form Medical Visual Question Answering in Radiology
Visual Question Answering (VQA) in the medical domain presents a unique, interdisciplinary challenge, combining fields such as Computer Vision, Natural Language Processing, and Knowledge Representation. Despite its impor…
DiagnosticFormMedical Visual Question AnsweringQuestion Answering+2FingerNet: Pushing The Limits of Fingerprint Recognition Using Convolutional Neural Network
Fingerprint recognition has been utilized for cellphone authentication, airport security and beyond. Many different features and algorithms have been proposed to improve fingerprint recognition. In this paper, we propose…