paper-with-me

Papers

Mamba Hawkes Process

2024-07-07 · Anningzhe Gao, Shan Dai, Yan Hu

Irregular and asynchronous event sequences are prevalent in many domains, such as social media, finance, and healthcare. Traditional temporal point processes (TPPs), like Hawkes processes, often struggle to model mutual inhibition and nonlinearity effectively. While recent neural network models, including RNNs and Transformers, address some of these issues, they still face challenges with long-term dependencies and computational efficiency. In this paper, we introduce the Mamba Hawkes Process (MHP), which leverages the Mamba state space architecture to capture long-range dependencies and dynamic event interactions. Our results show that MHP outperforms existing models across various datasets. Additionally, we propose the Mamba Hawkes Process Extension (MHP-E), which combines Mamba and Transformer models to enhance predictive capabilities. We present the novel application of the Mamba architecture to Hawkes processes, a flexible and extensible model structure, and a theoretical analysis of the synergy between state space models and Hawkes processes. Experimental results demonstrate the superior performance of both MHP and MHP-E, advancing the field of temporal point process modeling.

📄 PDF Abstract BibTeX arXiv:2407.05302

Code (0)

등록된 구현이 없습니다.

Tasks

Computational EfficiencyMambaPoint ProcessesState Space Models

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Multi-Head Attention 설명 없음
Attention 설명 없음
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Similar Papers 제목 키워드 기반

Non-Parametric Estimation of Multi-dimensional Marked Hawkes Processes

2024-02-07 · Sobin Joseph, Shashi Jain

An extension of the Hawkes process, the Marked Hawkes process distinguishes itself by featuring variable jump size across each event, in contrast to the constant jump size observed in a Hawkes process without marks. Whil…

Variance-Hawkes Process and its Application to Energy Markets

2024-10-10 · Joshua McGillivray, Anatoliy Swishchuk

We define a new model using a Hawkes process as a subordinator in a standard Brownian motion. We demonstrate that this Hawkes subordinated Brownian motion or more succinctly, variance-Hawkes process can be fit to 2018 an…

MUFM: A Mamba-Enhanced Feedback Model for Micro Video Popularity Prediction

2024-11-23 · Jiacheng Lu, Mingyuan Xiao, Weijian Wang, Yuxin Du 외

The surge in micro-videos is transforming the concept of popularity. As researchers delve into vast multi-modal datasets, there is a growing interest in understanding the origins of this popularity and the forces driving…

Mamba

Continuous-Time Reinforcement Learning for Controlled Hawkes Jump-Diffusions

2026-08-19 · Tomasz R. Bielecki, Thibaut Mastrolia, Haoze Yan arxiv

We study stochastic control of multivariate Hawkes-driven stochastic differential equations with machine learning algorithms in a non-Markovian setting. Due to the path dependence of the memory of the Hawkes intensity, t…

Reinforcement Learning

Application of Hawkes volatility in the observation of filtered high-frequency price process in tick structures

2022-07-13 · Kyungsub Lee

The Hawkes model is suitable for describing self and mutually exciting random events. In addition, the exponential decay in the Hawkes process allows us to calculate the moment properties in the model. However, due to th…

Management