Distributed Function Minimization in Apache Spark
We report on an open-source implementation for distributed function minimization on top of Apache Spark by using gradient and quasi-Newton methods. We show-case it with an application to Optimal Transport and some scalability tests on classification and regression problems.
Code (1)
Tasks
General ClassificationregressionSimilar Papers 제목 키워드 기반
Distributed Record Linkage in Healthcare Data with Apache Spark
Healthcare data is a valuable resource for research, analysis, and decision-making in the medical field. However, healthcare data is often fragmented and distributed across various sources, making it challenging to combi…
Decision MakingMMLSpark: Unifying Machine Learning Ecosystems at Massive Scales
We introduce Microsoft Machine Learning for Apache Spark (MMLSpark), an ecosystem of enhancements that expand the Apache Spark distributed computing library to tackle problems in Deep Learning, Micro-Service Orchestratio…
BIG-bench Machine LearningDistributed Computingobject-detectionObject DetectionBigDL: A Distributed Deep Learning Framework for Big Data
This paper presents BigDL (a distributed deep learning framework for Apache Spark), which has been used by a variety of users in the industry for building deep learning applications on production big data platforms. It a…
Deep LearningFraud DetectionManagementObject DetectionMLlib: Machine Learning in Apache Spark
Apache Spark is a popular open-source platform for large-scale data processing that is well-suited for iterative machine learning tasks. In this paper we present MLlib, Spark's open-source distributed machine learning li…
BIG-bench Machine LearningPerformance Evaluation of Query Plan Recommendation with Apache Hadoop and Apache Spark
Access plan recommendation is a query optimization approach that executes new queries using prior created query execution plans (QEPs). The query optimizer divides the query space into clusters in the mentioned method. H…
ClusteringDistributed Computing