Knowledge Graph and Hypergraph Transformers with Repository-Attention and Journey-Based Role Transport
We present a concise architecture for joint training on sentences and structured data while keeping knowledge and language representations separable. The model treats knowledge graphs and hypergraphs as structured instances with role slots and encodes them into a key-value repository that a language transformer can attend over. Attention is conditioned by journey-based role transport, which unifies edge-labeled KG traversal, hyperedge traversal, and sentence structure. We outline a dual-stream architecture, hierarchical layer groups with instance-local, neighborhood, and global mixing attention, retrieval over a separate repository, and multi-task objectives spanning masked language modeling, link prediction, and role-consistency denoising. The result is an explicit, inspectable separation between linguistic context and structured knowledge, while still enabling tight alignment through cross-attention.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge GraphsLink PredictionSimilar Papers 제목 키워드 기반
Transformers Generalize DeepSets and Can be Extended to Graphs & Hypergraphs
We present a generalization of Transformers to any-order permutation invariant data (sets, graphs, and hypergraphs). We begin by observing that Transformers generalize DeepSets, or first-order (set-input) permutation inv…
2kGraph RegressionTransformers Generalize DeepSets and Can be Extended to Graphs and Hypergraphs
We present a generalization of Transformers to any-order permutation invariant data (sets, graphs, and hypergraphs). We begin by observing that Transformers generalize DeepSets, or first-order (set-input) permutation inv…
2kGraph RegressionHyperedge PredictionSet-to-Graph PredictionHEAT: Hyperedge Attention Networks
Learning from structured data is a core machine learning task. Commonly, such data is represented as graphs, which normally only consider (typed) binary relationships between pairs of nodes. This is a substantial limitat…
Knowledge Base CompletionDynamic Hypergraph Representation Learning for Multivariate Time Series without Prior Knowledge
Hypergraphs have the capacity to capture higher-dimensional relationships among entities across various domains, making them a subject of growing interest within the research community for understanding the structure and…
Representation LearningCommunity DetectionNeural Algorithmic Reasoning for Hypergraphs with Looped Transformers
Looped Transformers have shown exceptional neural algorithmic reasoning capability in simulating traditional graph algorithms, but their application to more complex structures like hypergraphs remains underexplored. Hype…
Combinatorial Optimization