Two Architectures for Parallel Processing of Huge Amounts of Text
This paper presents two alternative NLP architectures to analyze massive amounts of documents, using parallel processing. The two architectures focus on different processing scenarios, namely batch-processing and streaming processing. The batch-processing scenario aims at optimizing the overall throughput of the system, i.e., minimizing the overall time spent on processing all documents. The streaming architecture aims to minimize the time to process real-time incoming documents and is therefore especially suitable for live feeds. The paper presents experiments with both architectures, and reports the overall gain when they are used for batch as well as for streaming processing. All the software described in the paper is publicly available under free licenses.
Code (0)
등록된 구현이 없습니다.
Tasks
Vocal Bursts Valence PredictionSimilar Papers 제목 키워드 기반
NTUA-ISLab at SemEval-2019 Task 3: Determining emotions in contextual conversations with deep learning
Sentiment analysis (SA) in texts is a well-studied Natural Language Processing task, which in nowadays gains popularity due to the explosion of social media, and the subsequent accumulation of huge amounts of related dat…
Sentiment AnalysisMaximizing Parallelism in Distributed Training for Huge Neural Networks
The recent Natural Language Processing techniques have been refreshing the state-of-the-art performance at an incredible speed. Training huge language models is therefore an imperative demand in both industry and academy…
Unsupervised Network Anomaly Detection with Autoencoders and Traffic Images
Due to the recent increase in the number of connected devices, the need to promptly detect security issues is emerging. Moreover, the high number of communication flows creates the necessity of processing huge amounts of…
Anomaly DetectionDistilling BERT into Simple Neural Networks with Unlabeled Transfer Data
Recent advances in pre-training huge models on large amounts of text through self supervision have obtained state-of-the-art results in various natural language processing tasks. However, these huge and expensive models …
Knowledge DistillationNERDAML-ST5: Low Resource Style Transfer via Domain Adaptive Meta Learning
Text style transfer (TST) without parallel data has achieved some practical success. However, most of the existing unsupervised text style transfer methods suffer from (i) requiring massive amounts of nonparallel data to…
General KnowledgeLanguage ModelingLanguage ModellingMeta-Learning+3