Self-Attention Mechanism in Multimodal Context for Banking Transaction Flow
Banking Transaction Flow (BTF) is a sequential data found in a number of banking activities such as marketing, credit risk or banking fraud. It is a multimodal data composed of three modalities: a date, a numerical value and a wording. We propose in this work an application of self-attention mechanism to the processing of BTFs. We trained two general models on a large amount of BTFs in a self-supervised way: one RNN-based model and one Transformer-based model. We proposed a specific tokenization in order to be able to process BTFs. The performance of these two models was evaluated on two banking downstream tasks: a transaction categorization task and a credit risk task. The results show that fine-tuning these two pre-trained models allowed to perform better than the state-of-the-art approaches for both tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
MarketingSimilar Papers 제목 키워드 기반
Incomplete Multimodal Learning for Remote Sensing Data Fusion
The mechanism of connecting multimodal signals through self-attention operation is a key factor in the success of multimodal Transformer networks in remote sensing data fusion tasks. However, traditional approaches assum…
Semantic SegmentationCSAT‑FTCN: A Fuzzy‑Oriented Model with Contextual Self‑attention Network for Multimodal Emotion Recognition
Multimodal emotion analysis has become a hot trend because of its wide applications, such as the question-answering system. However, in a real-world scenario, people usually have mixed or partial emotions about evaluati…
Emotion RecognitionMultimodal Emotion RecognitionQuestion AnsweringOpen Banking Foundational Model: Learning Language Representations from Few Financial Transactions
We introduced a multimodal foundational model for financial transactions that integrates both structured attributes and unstructured textual descriptions into a unified representation. By adapting masked language modelin…
Feature EngineeringMultimodal Sentiment Analysis based on Multi-channel and Symmetric Mutual Promotion Feature Fusion
Multimodal sentiment analysis is a key technology in the fields of human-computer interaction and affective computing. Accurately recognizing human emotional states is crucial for facilitating smooth communication betwee…
Multimodal Sentiment AnalysisMultimodal Document Analytics for Banking Process Automation
Traditional banks face increasing competition from FinTechs in the rapidly evolving financial ecosystem. Raising operational efficiency is vital to address this challenge. Our study aims to improve the efficiency of docu…
token-classificationToken Classification