Performance prediction of data streams on high-performance architecture

2019-01-07 · Human-centric Computing and Information Sciences 2019 1 · Bhaskar Gautam, Annappa Basava

Worldwide sensor streams are expanding continuously with unbounded velocity in volume, and for this acceleration, there is an adaptation of large stream data processing system from the homogeneous to rack-scale architecture which makes serious concern in the domain of workload optimization, scheduling, and resource management algorithms. Our proposed framework is based on providing architecture independent performance prediction model to enable resource adaptive distributed stream data processing platform. It is comprised of seven pre-defined domain for dynamic data stream metrics including a self-driven model which tries to fit these metrics using ridge regularization regression algorithm. Another significant contribution lies in fully-automated performance prediction model inherited from the state-of-the-art distributed data management system for distributed stream processing systems using Gaussian processes regression that cluster metrics with the help of dimensionality reduction algorithm. We implemented its base on Apache Heron and evaluated with proposed Benchmark Suite comprising of five domain-specific topologies. To assess the proposed methodologies, we forcefully ingest tuple skewness among the benchmarking topologies to set up the ground truth for predictions and found that accuracy of predicting the performance of data streams increased up to 80.62% from 66.36% along with the reduction of error from 37.14 to 16.06%.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Benchmarking Dimensionality Reduction Gaussian Processes Management Prediction regressionSchedulingVocal Bursts Intensity Prediction

Fast Mining and Dynamic Time-to-Event Prediction over Multi-sensor Data Streams

2026-01-08 · Kota Nakamura, Koki Kawabata, Yasuko Matsubara, Yasushi Sakurai arxiv

Given real-time sensor data streams obtained from machines, how can we continuously predict when a machine failure will occur? This work aims to continuously forecast the timing of future events by analyzing multi-sensor…

Prediction-Oriented Subsampling from Data Streams

2025-08-05 · Benedetta Lavinia Mussati, Freddie Bickford Smith, Tom Rainforth, Stephen Roberts arxiv

Data is often generated in streams, with new observations arriving over time. A key challenge for learning models from data streams is capturing relevant information while keeping computational costs manageable. We explo…

Cell Grid Architecture for Maritime Route Prediction on AIS Data Streams

2018-09-28 · Ciprian Amariei, Paul Diac, Emanuel Onica, Valentin Roşca

The 2018 Grand Challenge targets the problem of accurate predictions on data streams produced by automatic identification system (AIS) equipment, describing naval traffic. This paper reports the technical details of a cu…

Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams

2020-09-02 · ICCV 2021 10 · Matthias De Lange, Tinne Tuytelaars

Attaining prototypical features to represent class distributions is well established in representation learning. However, learning prototypes online from streaming data proves a challenging endeavor as they rapidly becom…

class-incremental learning Class Incremental Learning Continual Learning Incremental Learning+1

Cross-Representation Benchmarking in Time-Series Electronic Health Records for Clinical Outcome Prediction

2025-10-10 · Tianyi Chen, Mingcheng Zhu, Zhiyao Luo, Tingting Zhu arxiv

Electronic Health Records (EHRs) enable deep learning for clinical predictions, but the optimal method for representing patient data remains unclear due to inconsistent evaluation practices. We present the first systemat…