paper-with-me

Papers

Modality-Independent Graph Neural Networks with Global Transformers for Multimodal Recommendation

2024-12-18 · Jun Hu, Bryan Hooi, Bingsheng He, Yinwei Wei

Multimodal recommendation systems can learn users' preferences from existing user-item interactions as well as the semantics of multimodal data associated with items. Many existing methods model this through a multimodal user-item graph, approaching multimodal recommendation as a graph learning task. Graph Neural Networks (GNNs) have shown promising performance in this domain. Prior research has capitalized on GNNs' capability to capture neighborhood information within certain receptive fields (typically denoted by the number of hops, $K$) to enrich user and item semantics. We observe that the optimal receptive fields for GNNs can vary across different modalities. In this paper, we propose GNNs with Modality-Independent Receptive Fields, which employ separate GNNs with independent receptive fields for different modalities to enhance performance. Our results indicate that the optimal $K$ for certain modalities on specific datasets can be as low as 1 or 2, which may restrict the GNNs' capacity to capture global information. To address this, we introduce a Sampling-based Global Transformer, which utilizes uniform global sampling to effectively integrate global information for GNNs. We conduct comprehensive experiments that demonstrate the superiority of our approach over existing methods. Our code is publicly available at https://github.com/CrawlScript/MIG-GT.

📄 PDF Abstract BibTeX arXiv:2412.13994

Code (1)

crawlscript/mig-gt 공식 구현 pytorch

Tasks

Graph LearningMulti-modal RecommendationMultimodal RecommendationRecommendation Systems

Methods 이 논문이 사용한 방법론

Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Multi-Head Attention 설명 없음
Adam 설명 없음
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Position-Wise Feed-Forward Layer 설명 없음
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Similar Papers 제목 키워드 기반

Expressive Power of Graph Transformers via Logic

2025-08-01 · Veeti Ahvonen, Maurice Funk, Damian Heiman, Antti Kuusisto 외 arxiv

Transformers are the basis of modern large language models, but relatively little is known about their precise expressive power on graphs. We study the expressive power of graph transformers (GTs) by Dwivedi and Bresson …

A Modality-Aware Cooperative Co-Evolutionary Framework for Multimodal Graph Neural Architecture Search

2025-09-23 · Sixuan Wang, Jiao Yin, Jinli Cao, Mingjian Tang 외 arxiv

Co-exploitation attacks on software vulnerabilities pose severe risks to enterprises, a threat that can be mitigated by analyzing heterogeneous and multimodal vulnerability data. Multimodal graph neural networks (MGNNs) …

Neural Architecture Search

CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers

2023-05-27 · Dachuan Shi, Chaofan Tao, Anyi Rao, Zhendong Yang 외

Recent vision-language models have achieved tremendous advances. However, their computational costs are also escalating dramatically, making model acceleration exceedingly critical. To pursue more efficient vision-langua…

Image CaptioningImage RetrievalImage-text RetrievalImage-to-Text Retrieval+5

LGMRec: Local and Global Graph Learning for Multimodal Recommendation

2023-12-27 · Zhiqiang Guo, Jianjun Li, GuoHui Li, Chaoyang Wang 외

The multimodal recommendation has gradually become the infrastructure of online media platforms, enabling them to provide personalized service to users through a joint modeling of user historical behaviors (e.g., purchas…

Graph EmbeddingGraph Learninghypergraph embeddingMultimodal Recommendation

Multimodal Motion Prediction with Stacked Transformers

2021-03-22 · CVPR 2021 1 · Yicheng Liu, Jinghuai Zhang, Liangji Fang, Qinhong Jiang 외

Predicting multiple plausible future trajectories of the nearby vehicles is crucial for the safety of autonomous driving. Recent motion prediction approaches attempt to achieve such multimodal motion prediction by implic…

Autonomous DrivingDiversitymotion predictionPrediction