paper-with-me

Papers

BenchML: an extensible pipelining framework for benchmarking representations of materials and molecules at scale

2021-12-04 · Carl Poelking, Felix A. Faber, Bingqing Cheng

We introduce a machine-learning (ML) framework for high-throughput benchmarking of diverse representations of chemical systems against datasets of materials and molecules. The guiding principle underlying the benchmarking approach is to evaluate raw descriptor performance by limiting model complexity to simple regression schemes while enforcing best ML practices, allowing for unbiased hyperparameter optimization, and assessing learning progress through learning curves along series of synchronized train-test splits. The resulting models are intended as baselines that can inform future method development, next to indicating how easily a given dataset can be learnt. Through a comparative analysis of the training outcome across a diverse set of physicochemical, topological and geometric representations, we glean insight into the relative merits of these representations as well as their interrelatedness.

📄 PDF Abstract BibTeX arXiv:2112.02287

Code (2)

bingqingcheng/linear-regression-benchmarks 공식 구현
capoe/benchml 공식 구현

Tasks

BenchmarkingHyperparameter Optimizationregression

Similar Papers 제목 키워드 기반

A Benchmarking Framework for AI models in Automotive Aerodynamics

2025-07-14 · Kaustubh Tangsali, Rishikesh Ranade, Mohammad Amin Nabian, Alexey Kamenev 외 arxiv

In this paper, we introduce a benchmarking framework within the open-source NVIDIA PhysicsNeMo-CFD framework designed to systematically assess the accuracy, performance, scalability, and generalization capabilities of AI…

Optimal Software Pipelining and Warp Specialization for Tensor Core GPUs

2025-12-19 · Rupanshu Soi, Rohan Yadav, Fredrik Kjolstad, Alex Aiken 외 arxiv

GPU architectures have continued to grow in complexity, with recent incarnations introducing increasingly powerful fixed-function units for matrix multiplication and data movement to accompany highly parallel general-pur…

OSMa-Bench++: Toward Open-Ended Benchmarking of Semantic Mapping for Manipulation with Prompt-Generated Synthetic Scenes

2026-05-26 · Regina Kurkova, Maxim Popov, Sergey Kolyubin arxiv

Semantic mapping methods are increasingly used as intermediate scene representations for downstream robotic reasoning and manipulation, yet their evaluation is still largely tied to fixed benchmark datasets with limited …

Asynchronous Batch Bayesian Optimization with Pipelining Evaluations for Experimental Resource$\unicode{x2013}$constrained Conditions

2024-12-05 · Yujin Taguchi, Yusuke Shibuya, Yusuke Hiki, Takashi Morikura 외

Bayesian optimization is efficient even with a small amount of data and is used in engineering and in science, including biology and chemistry. In Bayesian optimization, a parameterized model with an uncertainty is fitte…

Bayesian OptimizationBenchmarking

STM-Graph: A Python Framework for Spatio-Temporal Mapping and Graph Neural Network Predictions

2025-09-04 · Amirhossein Ghaffari, Huong Nguyen, Lauri Lovén, Ekaterina Gilman arxiv

Urban spatio-temporal data present unique challenges for predictive analytics due to their dynamic and complex nature. We introduce STM-Graph, an open-source Python framework that transforms raw spatio-temporal urban eve…

Graph Neural Network