PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling
Most existing slot filling models tend to memorize inherent patterns of entities and corresponding contexts from training data. However, these models can lead to system failure or undesirable outputs when being exposed to spoken language perturbation or variation in practice. We propose a perturbed semantic structure awareness transferring method for training perturbation-robust slot filling models. Specifically, we introduce two MLM-based training strategies to respectively learn contextual semantic structure and word distribution from unsupervised language perturbation corpus. Then, we transfer semantic knowledge learned from upstream training procedure into the original samples and filter generated data by consistency processing. These procedures aim to enhance the robustness of slot filling models. Experimental results show that our method consistently outperforms the previous basic methods and gains strong generalization while preventing the model from memorizing inherent patterns of entities and contexts.
Code (0)
등록된 구현이 없습니다.
Tasks
slot-fillingSlot FillingSimilar Papers 제목 키워드 기반
Pre-Trained LLM is a Semantic-Aware and Generalizable Segmentation Booster
With the advancement of Large Language Model (LLM) for natural language processing, this paper presents an intriguing finding: a frozen pre-trained LLM layer can process visual tokens for medical image segmentation tasks…
DecoderImage SegmentationLanguage ModelingLanguage Modelling+4The OPS-SAT benchmark for detecting anomalies in satellite telemetry
Detecting anomalous events in satellite telemetry is a critical task in space operations. This task, however, is extremely time-consuming, error-prone and human dependent, thus automated data-driven anomaly detection alg…
Anomaly DetectionHASSL: Hierarchy-Aware Self-Supervised Learning Framework for Single Cell Microscopy
Hierarchical structure is common in image data, where fine-grained clusters often merge into larger, coarser semantic groups. In biological cell images, current self-supervised learning models often suppress this hierarc…
Self-Supervised LearningDense Semantic Matching with VGGT Prior
Semantic matching aims to establish pixel-level correspondences between instances of the same category and represents a fundamental task in computer vision. Existing approaches suffer from two limitations: (i) Geometric …
Data AugmentationCP-Net: Contour-Perturbed Reconstruction Network for Self-Supervised Point Cloud Learning
Self-supervised learning has not been fully explored for point cloud analysis. Current frameworks are mainly based on point cloud reconstruction. Given only 3D coordinates, such approaches tend to learn local geometric s…
Point cloud reconstructionSelf-Supervised Learning