Making Classical Machine Learning Pipelines Differentiable: A Neural Translation Approach
Classical Machine Learning (ML) pipelines often comprise of multiple ML models where models, within a pipeline, are trained in isolation. Conversely, when training neural network models, layers composing the neural models are simultaneously trained using backpropagation. We argue that the isolated training scheme of ML pipelines is sub-optimal, since it cannot jointly optimize multiple components. To this end, we propose a framework that translates a pre-trained ML pipeline into a neural network and fine-tunes the ML models within the pipeline jointly using backpropagation. Our experiments show that fine-tuning of the translated pipelines is a promising technique able to increase the final accuracy.
Code (1)
Tasks
BIG-bench Machine LearningTranslationSimilar Papers 제목 키워드 기반
Open Problems in Differentiable Social Choice: Learning Mechanisms, Decisions, and Alignment
Social choice has become a foundational component of modern machine learning systems. From auctions and resource allocation to the alignment of large generative models, machine learning pipelines increasingly aggregate h…
Bi-Directional Differentiable Input Reconstruction for Low-Resource Neural Machine Translation
We aim to better exploit the limited amounts of parallel text available in low-resource settings by introducing a differentiable reconstruction loss for neural machine translation (NMT). This loss compares original input…
Low Resource Neural Machine TranslationLow-Resource Neural Machine TranslationMachine TranslationNMT+1Reinforced Pipeline Optimization: Behaving Optimally with Non-Differentiabilities
Many machine learning systems are implemented as pipelines. A pipeline is essentially a chain/network of information processing units. As information flows in and out and gradients vice versa, ideally, a pipeline can be …
object-detectionObject DetectionVine Copulas as Differentiable Computational Graphs
Vine copulas are sophisticated models for multivariate distributions and are increasingly used in machine learning. To facilitate their integration into modern ML pipelines, we introduce the vine computational graph, a D…
GPUSchedulingUncertainty QuantificationATHAR: A High-Quality and Diverse Dataset for Classical Arabic to English Translation
Classical Arabic represents a significant era, encompassing the golden age of Arab culture, philosophy, and scientific literature. With a broad consensus on the importance of translating these literatures to enrich knowl…
PhilosophyTranslation