Feature selection in high-dimensional dataset using MapReduce
This paper describes a distributed MapReduce implementation of the minimum Redundancy Maximum Relevance algorithm, a popular feature selection method in bioinformatics and network inference problems. The proposed approach handles both tall/narrow and wide/short datasets. We further provide an open source implementation based on Hadoop/Spark, and illustrate its scalability on datasets involving millions of observations or features.
Code (1)
Tasks
feature selectionVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Polarization Measurement of High Dimensional Social Media Messages With Support Vector Machine Algorithm Using Mapreduce
In this article, we propose a new Support Vector Machine (SVM) training algorithm based on distributed MapReduce technique. In literature, there are a lots of research that shows us SVM has highest generalization propert…
ClassificationGeneral ClassificationSentiment AnalysisEmbed and Conquer: Scalable Embeddings for Kernel k-Means on MapReduce
The kernel $k$-means is an effective method for data clustering which extends the commonly-used $k$-means algorithm to work on a similarity matrix over complex data structures. The kernel $k$-means algorithm is however c…
ClusteringDistributed ComputingLLM$\times$MapReduce-V2: Entropy-Driven Convolutional Test-Time Scaling for Generating Long-Form Articles from Extremely Long Resources
Long-form generation is crucial for a wide range of practical applications, typically categorized into short-to-long and long-to-long generation. While short-to-long generations have received considerable attention, gene…
ArticlesFormA MapReduce based distributed SVM algorithm for binary classification
Although Support Vector Machine (SVM) algorithm has a high generalization property to classify for unseen examples after training phase and it has small loss value, the algorithm is not suitable for real-life classificat…
Binary ClassificationClassificationCloud ComputingGeneral Classification+1Distributed Correlation-Based Feature Selection in Spark
CFS (Correlation-Based Feature Selection) is an FS algorithm that has been successfully applied to classification problems in many domains. We describe Distributed CFS (DiCFS) as a completely redesigned, scalable, parall…
feature selection