Self-supervised Hypergraphs for Learning Multiple World Interpretations
We present a method for learning multiple scene representations given a small labeled set, by exploiting the relationships between such representations in the form of a multi-task hypergraph. We also show how we can use the hypergraph to improve a powerful pretrained VisTransformer model without any additional labeled data. In our hypergraph, each node is an interpretation layer (e.g., depth or segmentation) of the scene. Within each hyperedge, one or several input nodes predict the layer at the output node. Thus, each node could be an input node in some hyperedges and an output node in others. In this way, multiple paths can reach the same node, to form ensembles from which we obtain robust pseudolabels, which allow self-supervised learning in the hypergraph. We test different ensemble models and different types of hyperedges and show superior performance to other multi-task graph models in the field. We also introduce Dronescapes, a large video dataset captured with UAVs in different complex real-world scenes, with multiple representations, suitable for multi-task learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Multi-Task LearningSelf-Supervised LearningSimilar Papers 제목 키워드 기반
HypeBoy: Generative Self-Supervised Representation Learning on Hypergraphs
Hypergraphs are marked by complex topology, expressing higher-order interactions among multiple nodes with hyperedges, and better capturing the topology is essential for effective representation learning. Recent advances…
Hypergraph representationsNode ClassificationRepresentation LearningSelf-Supervised LearningBHyGNN+: Unsupervised Representation Learning for Heterophilic Hypergraphs
Hypergraph Neural Networks (HyGNNs) have demonstrated remarkable success in modeling higher-order relationships among entities. However, their performance often degrades on heterophilic hypergraphs, where nodes connected…
Hypergraph Contrastive LearningSelf-Supervised LearningRepresentation LearningHypergraph Self-supervised Learning with Sampling-efficient Signals
Self-supervised learning (SSL) provides a promising alternative for representation learning on hypergraphs without costly labels. However, existing hypergraph SSL models are mostly based on contrastive methods with the i…
Representation LearningSelf-Supervised LearningHyper-SAGNN: a self-attention based graph neural network for hypergraphs
Graph representation learning for hypergraphs can be used to extract patterns among higher-order interactions that are critically important in many real world problems. Current approaches designed for hypergraphs, howeve…
Graph Neural NetworkGraph Representation LearningLink PredictionRepresentation LearningSelf-Supervised Pretraining for Heterogeneous Hypergraph Neural Networks
Recently, pretraining methods for the Graph Neural Networks (GNNs) have been successful at learning effective representations from unlabeled graph data. However, most of these methods rely on pairwise relations in the gr…
Link PredictionNode Classification