paper-with-me

홈 › Papers

STKDRec: Spatial-Temporal Knowledge Distillation for Takeaway Recommendation

2024-12-21 · Shuyuan Zhao, Wei Chen, Boyan Shi, Liyong Zhou, Shuohao Lin, Huaiyu Wan

The takeaway recommendation system is designed to recommend users' future takeaway purchases based on their historical purchase behaviors, thereby improving user satisfaction and increasing merchant sales. Existing methods focus on incorporating auxiliary information or leveraging knowledge graphs to alleviate the sparsity issue of user purchase sequence data. However, two main challenges limit the performance of these approaches: (1) how to capture dynamic user preferences on complex geospatial information and (2) how to efficiently integrate spatial-temporal knowledge from graphs and sequence data with low calculation costs. In this paper, we propose a novel spatial-temporal knowledge distillation for takeaway recommendation model (STKDRec) based on the two-stage training process. Specifically, during the first pre-training stage, a spatial-temporal knowledge graph (STKG) encoder is pre-trained to extract the high-order spatial-temporal and collaborative associations within the STKG. During the second STKD stage, a spatial-temporal Transformer is employed to comprehensively model dynamic user preferences on various types of fine-grained geospatial information from a sequence perspective. Furthermore, the STKD strategy is introduced to adaptively fuse the rich spatial-temporal knowledge from the pre-trained STKG encoder and the spatial-temporal transformer while reducing the cost of model training. Extensive experiments on three real-world datasets show that our STKDRec significantly outperforms the state-of-the-art baselines. Our code is available at:https://github.com/Zhaoshuyuan0246/STKDRec.

📄 PDF Abstract BibTeX arXiv:2412.16502

Code (1)

zhaoshuyuan0246/stkdrec 공식 구현 pytorch

Tasks

Knowledge DistillationKnowledge Graphs

Methods 이 논문이 사용한 방법론

Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…
Multi-Head Attention 설명 없음
Position-Wise Feed-Forward Layer 설명 없음

Similar Papers 제목 키워드 기반

Fast Video Salient Object Detection via Spatiotemporal Knowledge Distillation

2020-10-20 · Yi Tang, Yuanman Li, Wenbin Zou

Since the wide employment of deep learning frameworks in video salient object detection, the accuracy of the recent approaches has made stunning progress. These approaches mainly adopt the sequential modules, based on op…

Knowledge DistillationObjectobject-detectionObject Detection+3

SCJD: Sparse Correlation and Joint Distillation for Efficient 3D Human Pose Estimation

2025-03-18 · Weihong Chen, Xuemiao Xu, Haoxin Yang, Yi Xie 외

Existing 3D Human Pose Estimation (HPE) methods achieve high accuracy but suffer from computational overhead and slow inference, while knowledge distillation methods fail to address spatial relationships between joints a…

3D Human Pose EstimationKnowledge DistillationPose EstimationTransfer Learning

COMEDIAN: Self-Supervised Learning and Knowledge Distillation for Action Spotting using Transformers

2023-09-03 · Julien Denize, Mykola Liashuha, Jaonary Rabarisoa, Astrid Orcesi 외

We present COMEDIAN, a novel pipeline to initialize spatiotemporal transformers for action spotting, which involves self-supervised learning and knowledge distillation. Action spotting is a timestamp-level temporal actio…

Action DetectionAction SpottingKnowledge DistillationSelf-Supervised Learning

STXD: Structural and Temporal Cross-Modal Distillation for Multi-View 3D Object Detection

2023-09-21 · NeurIPS 2023 11

3D object detection (3DOD) from multi-view images is an economically appealing alternative to expensive LiDAR-based detectors, but also an extremely challenging task due to the absence of precise spatial cues. Recent stu…

Efficient Traffic Prediction Through Spatio-Temporal Distillation

2025-01-15 · Qianru Zhang, Xinyi Gao, Haixin Wang, Siu-Ming Yiu 외

Graph neural networks (GNNs) have gained considerable attention in recent years for traffic flow prediction due to their ability to learn spatio-temporal pattern representations through a graph-based message-passing fram…

Knowledge DistillationPredictionTraffic Prediction