Local Structure Matters Most: Perturbation Study in NLU
Recent research analyzing the sensitivity of natural language understanding models to word-order perturbations has shown that neural models are surprisingly insensitive to the order of words. In this paper, we investigate this phenomenon by developing order-altering perturbations on the order of words, subwords, and characters to analyze their effect on neural models' performance on language understanding tasks. We experiment with measuring the impact of perturbations to the local neighborhood of characters and global position of characters in the perturbed texts and observe that perturbation functions found in prior literature only affect the global ordering while the local ordering remains relatively unperturbed. We empirically show that neural models, invariant of their inductive biases, pretraining scheme, or the choice of tokenization, mostly rely on the local structure of text to build understanding and make limited use of the global structure.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language UnderstandingPositionSimilar Papers 제목 키워드 기반
Local Structure Matters Most: Perturbation Study in NLU
Recent research analyzing the sensitivity of natural language understanding models to word-order perturbations has shown that neural models are surprisingly insensitive to the order of words. In this paper, we investigat…
Natural Language UnderstandingPositionLocal Structure Matters Most in Most Languages
Many recent perturbation studies have found unintuitive results on what does and does not matter when performing Natural Language Understanding (NLU) tasks in English. Coding properties, such as the order of words, can o…
Natural Language UnderstandingFrozen DINO Localizes Image Edits Without a Localizer
Localized image edits can change a photograph's meaning while leaving most of it authentic, so forensic analysis must identify where an edit occurred. We show that patch-level perturbation responses from frozen DINO enco…
Topology Matters: A Cautionary Case Study of Graph SSL on Neuro-Inspired Benchmarks
Understanding how local interactions give rise to global brain organization requires models that can represent information across multiple scales. We introduce a hierarchical self-supervised learning (SSL) framework that…
Self-Supervised LearningWhy semantics matters: A deep study on semantic particle-filtering localization in a LiDAR semantic pole-map
In most urban and suburban areas, pole-like structures such as tree trunks or utility poles are ubiquitous. These structural landmarks are very useful for the localization of autonomous vehicles given their geometrical l…
Autonomous VehiclesSemantic Segmentation