Transformer Embeddings of Irregularly Spaced Events and Their Participants

2021-12-31 · ICLR 2022 4 · Chenghao Yang, Hongyuan Mei, Jason Eisner

The neural Hawkes process (Mei & Eisner, 2017) is a generative model of irregularly spaced sequences of discrete events. To handle complex domains with many event types, Mei et al. (2020a) further consider a setting in which each event in the sequence updates a deductive database of facts (via domain-specific pattern-matching rules); future events are then conditioned on the database contents. They show how to convert such a symbolic system into a neuro-symbolic continuous-time generative model, in which each database fact and the possible event has a time-varying embedding that is derived from its symbolic provenance. In this paper, we modify both models, replacing their recurrent LSTM-based architectures with flatter attention-based architectures (Vaswani et al., 2017), which are simpler and more parallelizable. This does not appear to hurt our accuracy, which is comparable to or better than that of the original models as well as (where applicable) previous attention-based methods (Zuo et al., 2020; Zhang et al., 2020a).

📄 PDF Abstract BibTeX arXiv:2201.00044

Code (2)

yangalan123/anhp-andtt 공식 구현 pytorch

ivan-chai/hotpp-benchmark pytorch

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

MEI MEI introduces the *multi-partition embedding interaction* technique with block term tensor format to systematically address the efficiency--expressiveness trade-off in…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Residual Connection 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Learning Longitudinal Stress Dynamics from Irregular Self-Reports via Time Embeddings

2025-09-01 · Louis Simon, Mohamed Chetouani arxiv

The widespread adoption of mobile and wearable sensing technologies has enabled continuous and personalized monitoring of affect, mood disorders, and stress. When combined with ecological self-report questionnaires, thes…

Kernels for time series with irregularly-spaced multivariate observations

2020-04-18 · Ahmed Guecioueur, Franz J. Király

Time series are an interesting frontier for kernel-based methods, for the simple reason that there is no kernel designed to represent them and their unique characteristics in full generality. Existing sequential kernels …

Time Series Time Series Analysis Time Series Classification

Sharp Lower Bounds on Interpolation by Deep ReLU Neural Networks at Irregularly Spaced Data

2023-02-02 · Jonathan W. Siegel

We study the interpolation power of deep ReLU neural networks. Specifically, we consider the question of how efficiently, in terms of the number of parameters, deep ReLU networks can interpolate values at $N$ datapoints …

Memorization

An Intraday GARCH Model for Discrete Price Changes and Irregularly Spaced Observations

2022-11-22 · Vladimír Holý

We develop a novel observation-driven model for high-frequency prices. We account for irregularly spaced observations, simultaneous transactions, discreteness of prices, and market microstructure noise. The relation betw…

Revealing the temporal dynamics of antibiotic anomalies in the infant gut microbiome with neural jump ODEs

2025-09-30 · Anja Adamov, Markus Chardonnet, Florian Krach, Jakob Heiss 외 arxiv

Detecting anomalies in irregularly sampled multi-variate time-series is challenging, especially in data-scarce settings. Here we introduce an anomaly detection framework for irregularly sampled time-series that leverages…

Anomaly Detection