Downstream use of syntactic analysis: does representation matter?

Because Syntax Does Matter: Improving Predicate-Argument Structures Parsing with Syntactic Features

"Let's Eat Grandma": Does Punctuation Matter in Sentence Representation?

Neural network-based embeddings have been the mainstream approach for creating a vector representation of the text to capture lexical and semantic similarities and dissimilarities. In general, existing encoding methods d…

Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little

A possible explanation for the impressive performance of masked language model (MLM) pre-training is that such models have learned to represent the syntactic structures prevalent in classical NLP pipelines. In this paper…

Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa

Aspect-based Sentiment Analysis (ABSA), aiming at predicting the polarities for aspects, is a fine-grained task in the field of sentiment analysis. Previous work showed syntactic information, e.g. dependency trees, can e…

Syntax-guided Contrastive Learning for Pre-trained Language Model

Syntactic information has been proved to be useful for transformer-based pre-trained language models. Previous studies often rely on additional syntax-guided attention components to enhance the transformer, which require…