Performance Evaluation of Query Plan Recommendation with Apache Hadoop and Apache Spark
Access plan recommendation is a query optimization approach that executes new queries using prior created query execution plans (QEPs). The query optimizer divides the query space into clusters in the mentioned method. However, traditional clustering algorithms take a significant amount of execution time for clustering such large datasets. The MapReduce distributed computing model provides efficient solutions for storing and processing vast quantities of data. Apache Spark and Apache Hadoop frameworks are used in the present investigation to cluster different sizes of query datasets in the MapReduce-based access plan recommendation method. The performance evaluation is performed based on execution time. The results of the experiments demonstrated the effectiveness of parallel query clustering in achieving high scalability. Furthermore, Apache Spark achieved better performance than Apache Hadoop, reaching an average speedup of 2x.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringDistributed ComputingSimilar Papers 제목 키워드 기반
Correspondence Factor Analysis of Big Data Sets: A Case Study of 30 Million Words; and Contrasting Analytics using Apache Solr and Correspondence Analysis in R
We consider a large number of text data sets. These are cooking recipes. Term distribution and other distributional properties of the data are investigated. Our aim is to look at various analytical approaches which allow…
Neighborhood Troubles: On the Value of User Pre-Filtering To Speed Up and Enhance Recommendations
In this paper, we present work-in-progress on applying user pre-filtering to speed up and enhance recommendations based on Collaborative Filtering. We propose to pre-filter users in order to extract a smaller set of cand…
Collaborative FilteringRecommendation SystemsApache Lucene as Content-Based-Filtering Recommender System: 3 Lessons Learned
For the past few years, we used Apache Lucene as recommendation frame-work in our scholarly-literature recommender system of the reference-management software Docear. In this paper, we share three lessons learned from ou…
ManagementRecommendation Systems"This Suits You the Best": Query Focused Comparative Explainable Summarization
Product recommendations inherently involve comparisons, yet traditional opinion summarization often fails to provide holistic comparative insights. We propose the novel task of generating Query-Focused Comparative Explai…
Large-Scale Network Embedding in Apache Spark
Network embedding has been widely used in social recommendation and network analysis, such as recommendation systems and anomaly detection with graphs. However, most of previous approaches cannot handle large graphs effi…
Anomaly DetectionLink PredictionNetwork EmbeddingNode Classification+1