Structure Representation Learning by Jointly Learning to Pool and Represent
Structure representation learning is a task to provide an overall representation for a given structure (e.g., sequential text, non-sequential graph). This representation characterizes the property of that structure. Previous methods decompose the task into an element representation learning phase and a pooling phase to aggregate element representations. Their pooling phase only considers the final representation of each element without considering the relationship between these elements that are used only to construct representations of elements. In this paper, we conjecture that classification performance suffers from the lack of relation exploitation while pooling and propose the Self-Attention Pooling to dynamically provide centrality scores for pooling based on the self-attention scores from the element representation learning. Simply applying Self-Attention Pooling improves model performance on $3$ sentence classification tasks ({$\boldsymbol{\uparrow 2.9}$}) and $5$ graph classification tasks ({$\boldsymbol{\uparrow 2.1}$}) on average.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGraph ClassificationRepresentation LearningSentenceSentence ClassificationSimilar Papers 제목 키워드 기반
Hierarchical Spherical CNNs with Lifting-based Adaptive Wavelets for Pooling and Unpooling
Pooling and unpooling are two essential operations in constructing hierarchical spherical convolutional neural networks (HS-CNNs) for comprehensive feature learning in the spherical domain. Most existing models employ do…
CommPOOL: An Interpretable Graph Pooling Framework for Hierarchical Graph Representation Learning
Recent years have witnessed the emergence and flourishing of hierarchical graph pooling neural networks (HGPNNs) which are effective graph representation learning approaches for graph level tasks such as graph classifica…
Graph ClassificationGraph Representation LearningRepresentation LearningDiscriminatively Learned Hierarchical Rank Pooling Networks
In this work, we present novel temporal encoding methods for action and activity classification by extending the unsupervised rank pooling temporal encoding method in two ways. First, we present "discriminative rank pool…
Activity RecognitionBilevel OptimizationGeneral ClassificationAdaJudge: Adaptive Multi-Perspective Judging for Reward Modeling
Reward modeling is essential for aligning large language models with human preferences, yet predominant architectures rely on a static pooling strategy to condense sequences into scalar scores. This paradigm, however, su…
Graph Convolutional Networks with EigenPooling
Graph neural networks, which generalize deep neural network models to graph structured data, have attracted increasing attention in recent years. They usually learn node representations by transforming, propagating and a…
ClassificationGeneral ClassificationGraph ClassificationGraph Neural Network+4