ViBERTgrid BiLSTM-CRF: Multimodal Key Information Extraction from Unstructured Financial Documents
Multimodal key information extraction (KIE) models have been studied extensively on semi-structured documents. However, their investigation on unstructured documents is an emerging research topic. The paper presents an approach to adapt a multimodal transformer (i.e., ViBERTgrid previously explored on semi-structured documents) for unstructured financial documents, by incorporating a BiLSTM-CRF layer. The proposed ViBERTgrid BiLSTM-CRF model demonstrates a significant improvement in performance (up to 2 percentage points) on named entity recognition from unstructured documents in financial domain, while maintaining its KIE performance on semi-structured documents. As an additional contribution, we publicly released token-level annotations for the SROIE dataset in order to pave the way for its use in multimodal sequence labeling models.
Code (0)
등록된 구현이 없습니다.
Tasks
Key Information Extractionnamed-entity-recognitionNamed Entity RecognitionSimilar Papers 제목 키워드 기반
ViBERTgrid: A Jointly Trained Multi-Modal 2D Document Representation for Key Information Extraction from Documents
Recent grid-based document representations like BERTgrid allow the simultaneous encoding of the textual and layout information of a document in a 2D feature map so that state-of-the-art image segmentation and/or object d…
Image SegmentationKey Information Extractionobject-detectionObject Detection+2Multimodal Learning for Cardiovascular Risk Prediction using EHR Data
Electronic health records (EHRs) contain structured and unstructured data of significant clinical and research value. Various machine learning approaches have been developed to employ information in EHRs for risk predict…
Word EmbeddingsGraph Convolution for Multimodal Information Extraction from Visually Rich Documents
Visually rich documents (VRDs) are ubiquitous in daily business and life. Examples are purchase receipts, insurance policy documents, custom declaration forms and so on. In VRDs, visual and layout information is critical…
document understandingEntity Extraction using GANMultimodal Attribute Extraction
The broad goal of information extraction is to derive structured information from unstructured data. However, most existing methods focus solely on text, ignoring other types of unstructured data such as images, video an…
AttributeAttribute ExtractionMultimodal Attribute Value ExtractionExploring Multimodal Sentiment Analysis via CBAM Attention and Double-layer BiLSTM Architecture
Because multimodal data contains more modal information, multimodal sentiment analysis has become a recent research hotspot. However, redundant information is easily involved in feature fusion after feature extraction, w…
Multimodal Sentiment AnalysisSentiment Analysis