Adversarial Validation Approach to Concept Drift Problem in User Targeting Automation Systems at Uber
In user targeting automation systems, concept drift in input data is one of the main challenges. It deteriorates model performance on new data over time. Previous research on concept drift mostly proposed model retraining after observing performance decreases. However, this approach is suboptimal because the system fixes the problem only after suffering from poor performance on new data. Here, we introduce an adversarial validation approach to concept drift problems in user targeting automation systems. With our approach, the system detects concept drift in new data before making inference, trains a model, and produces predictions adapted to the new data. We show that our approach addresses concept drift effectively with the AutoML3 Lifelong Machine Learning challenge data as well as in Uber's internal user targeting automation system, MaLTA.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Handling Adversarial Concept Drift in Streaming Data
Classifiers operating in a dynamic, real world environment, are vulnerable to adversarial activity, which causes the data distribution to change over time. These changes are traditionally referred to as concept drift, an…
Active LearningDrift DetectionAdversarial Concept Drift Detection under Poisoning Attacks for Robust Data Stream Mining
Continuous learning from streaming data is among the most challenging topics in the contemporary machine learning. In this domain, learning algorithms must not only be able to handle massive volumes of rapidly arriving d…
Drift DetectionTackling Data Drift with Adversarial Validation: An Application for German Text Complexity Estimation
This paper describes the winning approach in the first automated German text complexity assessment shared task as part of KONVENS 2022. To solve this difficult problem, the evaluated system relies on an ensemble of regre…
Feature EngineeringregressionOptimized Deep Learning Models for Malware Detection under Concept Drift
Despite the promising results of machine learning models in malicious files detection, they face the problem of concept drift due to their constant evolution. This leads to declining performance over time, as the data di…
Malware DetectionDriftGAN: Using historical data for Unsupervised Recurring Drift Detection
In real-world applications, input data distributions are rarely static over a period of time, a phenomenon known as concept drift. Such concept drifts degrade the model's prediction performance, and therefore we require …
Drift Detection