Step by Step Loss Goes Very Far: Multi-Step Quantization for Adversarial Text Attacks
We propose a novel gradient-based attack against transformer-based language models that searches for an adversarial example in a continuous space of token probabilities. Our algorithm mitigates the gap between adversarial loss for continuous and discrete text representations by performing multi-step quantization in a quantization-compensation loop. Experiments show that our method significantly outperforms other approaches on various natural language processing (NLP) tasks.
Code (1)
Tasks
Adversarial TextQuantizationSimilar Papers 제목 키워드 기반
When to Re-Plan: Subgoal Persistence in Hierarchical Latent Reasoning
Long-horizon reasoning requires a system to commit to medium-horizon intent without becoming rigid: re-plan too often and computation never coheres into multi-step structure; commit too long and the plan goes stale. We s…
Finite-time optimality of Bayesian predictors
The problem of sequential probability forecasting is considered in the most general setting: a model set C is given, and it is required to predict as well as possible if any of the measures (environments) in C is chosen …
Towards Statistical and Computational Complexities of Polyak Step Size Gradient Descent
We study the statistical and computational complexities of the Polyak step size gradient descent algorithm under generalized smoothness and Lojasiewicz conditions of the population loss function, namely, the limit of the…
High-dimensional limit theorems for SGD: Effective dynamics and critical scaling
We study the scaling limits of stochastic gradient descent (SGD) with constant step-size in the high-dimensional regime. We prove limit theorems for the trajectories of summary statistics (i.e., finite-dimensional functi…
Vocal Bursts Intensity PredictionA labeled dataset of cloud types using data from GOES-16 and CloudSat
In this paper we present the development of a dataset consisting of 91 Multi-band Cloud and Moisture Product Full-Disk (MCMIPF) from the Advanced Baseline Imager (ABI) on board GOES-16 geostationary satellite with 91 tem…