paper-with-me

Papers

Optimal statistical inference in the presence of systematic uncertainties using neural network optimization based on binned Poisson likelihoods with nuisance parameters

2020-03-16 · Stefan Wunsch, Simon Jörger, Roger Wolf, Günter Quast

Data analysis in science, e.g., high-energy particle physics, is often subject to an intractable likelihood if the observables and observations span a high-dimensional input space. Typically the problem is solved by reducing the dimensionality using feature engineering and histograms, whereby the latter technique allows to build the likelihood using Poisson statistics. However, in the presence of systematic uncertainties represented by nuisance parameters in the likelihood, the optimal dimensionality reduction with a minimal loss of information about the parameters of interest is not known. This work presents a novel strategy to construct the dimensionality reduction with neural networks for feature engineering and a differential formulation of histograms so that the full workflow can be optimized with the result of the statistical inference, e.g., the variance of a parameter of interest, as objective. We discuss how this approach results in an estimate of the parameters of interest that is close to optimal and the applicability of the technique is demonstrated with a simple example based on pseudo-experiments and a more complex example from high-energy particle physics.

📄 PDF Abstract BibTeX arXiv:2003.07186

Code (0)

등록된 구현이 없습니다.

Tasks

Dimensionality ReductionFeature Engineering

Similar Papers 제목 키워드 기반

Stable but Wrong: An Inference Limit in Galactic Archaeology

2026-04-30 · Zhipeng Zhang arxiv

Statistical inference in observational science typically relies on a fundamental assumption: as sample size increases and uncertainties decrease, the inferred results should converge to the true physical quantities. This…

Data-Driven High-Dimensional Statistical Inference with Generative Models

2025-06-06 · Oz Amram, Manuel Szewc

Crucial to many measurements at the LHC is the use of correlated multi-dimensional information to distinguish rare processes from large backgrounds, which is complicated by the poor modeling of many of the crucial backgr…

Profiling systematic uncertainties in Simulation-Based Inference with Factorizable Normalizing Flows

2026-02-13 · Davide Valsecchi, Mauro Donegà, Rainer Wallny arxiv

Unbinned likelihood fits maximize the information extracted from experimental data, yet their application in realistic high-dimensional analyses has been fundamentally bottlenecked by the prohibitive computational cost o…

Generalization vs. Memorization in the Presence of Statistical Biases in Transformers

2024-09-06 · John Mitros, Damien Teney

This study aims to understand how statistical biases affect the model's ability to generalize to in-distribution and out-of-distribution data on algorithmic tasks. Prior research indicates that transformers may inadverte…

Memorization

Unifying supervised learning and VAEs -- coverage, systematics and goodness-of-fit in normalizing-flow based neural network models for astro-particle reconstructions

2020-08-13 · Thorsten Glüsenkamp

Neural-network based predictions of event properties in astro-particle physics are getting more and more common. However, in many cases the result is just utilized as a point prediction. Statistical uncertainties, covera…

Numerical IntegrationVariational Inference