Effective Parallelisation for Machine Learning
We present a novel parallelisation scheme that simplifies the adaptation of learning algorithms to growing amounts of data as well as growing needs for accurate and confident predictions in critical applications. In contrast to other parallelisation techniques, it can be applied to a broad class of learning algorithms without further mathematical derivations and without writing dedicated code, while at the same time maintaining theoretical performance guarantees. Moreover, our parallelisation scheme is able to reduce the runtime of many learning algorithms to polylogarithmic time on quasi-polynomially many processing units. This is a significant step towards a general answer to an open question on the efficient parallelisation of machine learning algorithms in the sense of Nick's Class (NC). The cost of this parallelisation is in the form of a larger sample complexity. Our empirical study confirms the potential of our parallelisation scheme with fixed numbers of processors and instances in realistic application scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningOpen-Ended Question AnsweringSimilar Papers 제목 키워드 기반
Batch Selection for Parallelisation of Bayesian Quadrature
Integration over non-negative integrands is a central problem in machine learning (e.g. for model averaging, (hyper-)parameter marginalisation, and computing posterior predictive distributions). Bayesian Quadrature is a …
Bayesian OptimisationBIG-bench Machine LearningNumerical IntegrationTriclustering in Big Data Setting
In this paper, we describe versions of triclustering algorithms adapted for efficient calculations in distributed environments with MapReduce model or parallelisation mechanism provided by modern programming languages. O…
Partitioning Distributed Compute Jobs with Reinforcement Learning and Graph Neural Networks
From natural language processing to genome sequencing, large-scale machine learning models are bringing advances to a broad range of fields. Many of these models are too large to be trained on a single machine, and inste…
BlockingGraph Neural NetworkManagementreinforcement-learning+1Parallelised Differentiable Straightest Geodesics for 3D Meshes
Machine learning has been progressively generalised to operate within non-Euclidean domains, but geometrically accurate methods for learning on surfaces are still falling behind. The lack of closed-form Riemannian operat…
Parallel Constraint-Driven Inductive Logic Programming
Multi-core machines are ubiquitous. However, most inductive logic programming (ILP) approaches use only a single core, which severely limits their scalability. To address this limitation, we introduce parallel techniques…
Inductive logic programmingProgram Synthesis