Ranking Robustness Under Adversarial Document Manipulations
For many queries in the Web retrieval setting there is an on-going ranking competition: authors manipulate their documents so as to promote them in rankings. Such competitions can have unwarranted effects not only in terms of retrieval effectiveness, but also in terms of ranking robustness. A case in point, rankings can (rapidly) change due to small indiscernible perturbations of documents. While there has been a recent growing interest in analyzing the robustness of classifiers to adversarial manipulations, there has not yet been a study of the robustness of relevance-ranking functions. We address this challenge by formally analyzing different definitions and aspects of the robustness of learning-to-rank-based ranking functions. For example, we formally show that increased regularization of linear ranking functions increases ranking robustness. This finding leads us to conjecture that decreased variance of any ranking function results in increased robustness. We propose several measures for quantifying ranking robustness and use them to analyze ranking competitions between documents' authors. The empirical findings support our formal analysis and conjecture for both RankSVM and LambdaMART.
Code (0)
등록된 구현이 없습니다.
Tasks
Learning-To-RankRetrievalSimilar Papers 제목 키워드 기반
Defense of Adversarial Ranking Attack in Text Retrieval: Benchmark and Baseline via Detection
Neural ranking models (NRMs) have undergone significant development and have become integral components of information retrieval (IR) systems. Unfortunately, recent research has unveiled the vulnerability of NRMs to adve…
Adversarial AttackInformation RetrievalLinguistic AcceptabilityRetrieval+1One word at a time: adversarial attacks on retrieval models
Adversarial examples, generated by applying small perturbations to input features, are widely used to fool classifiers and measure their robustness to noisy inputs. However, little work has been done to evaluate the robu…
RetrievalBERT Rankers are Brittle: a Study using Adversarial Document Perturbations
Contextual ranking models based on BERT are now well established for a wide range of passage and document ranking tasks. However, the robustness of BERT-based ranking models under adversarial inputs is under-explored. In…
Document RankingTowards Imperceptible Document Manipulations against Neural Ranking Models
Adversarial attacks have gained traction in order to identify potential vulnerabilities in neural ranking models (NRMs), but current attack methods often introduce grammatical errors, nonsensical expressions, or incohere…
Adversarial TextLanguage ModelingLanguage ModellingText GenerationCertified Robustness to Word Substitution Ranking Attack for Neural Ranking Models
Neural ranking models (NRMs) have achieved promising results in information retrieval. NRMs have also been shown to be vulnerable to adversarial examples. A typical Word Substitution Ranking Attack (WSRA) against NRMs wa…
Information RetrievalRetrieval