paper-with-me

Papers

Towards Learning Stochastic Population Models by Gradient Descent

2024-04-10 · Justin N. Kreikemeyer, Philipp Andelfinger, Adelinde M. Uhrmacher

Increasing effort is put into the development of methods for learning mechanistic models from data. This task entails not only the accurate estimation of parameters but also a suitable model structure. Recent work on the discovery of dynamical systems formulates this problem as a linear equation system. Here, we explore several simulation-based optimization approaches, which allow much greater freedom in the objective formulation and weaker conditions on the available data. We show that even for relatively small stochastic population models, simultaneous estimation of parameters and structure poses major challenges for optimization procedures. Particularly, we investigate the application of the local stochastic gradient descent method, commonly used for training machine learning models. We demonstrate accurate estimation of models but find that enforcing the inference of parsimonious, interpretable models drastically increases the difficulty. We give an outlook on how this challenge can be overcome.

📄 PDF Abstract BibTeX arXiv:2404.07049

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Analysis of Stochastic Gradient Descent in Continuous Time

2020-04-15 · Jonas Latz

Stochastic gradient descent is an optimisation method that combines classical gradient descent with random subsampling within the target functional. In this work, we introduce the stochastic gradient process as a continu…

Never Go Full Batch (in Stochastic Convex Optimization)

2021-06-29 · NeurIPS 2021 12 · Idan Amir, Yair Carmon, Tomer Koren, Roi Livni

We study the generalization performance of $\text{full-batch}$ optimization algorithms for stochastic convex optimization: these are first-order methods that only access the exact gradient of the empirical risk (rather t…

Acoustic Model Optimization Based On Evolutionary Stochastic Gradient Descent with Anchors for Automatic Speech Recognition

2019-07-10 · Xiaodong Cui, Michael Picheny

Evolutionary stochastic gradient descent (ESGD) was proposed as a population-based approach that combines the merits of gradient-aware and gradient-free optimization algorithms for superior overall optimization performan…

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Model Optimizationspeech-recognition+1

Evolutionary Stochastic Gradient Descent for Optimization of Deep Neural Networks

2018-10-16 · NeurIPS 2018 12 · Xiaodong Cui, Wei zhang, Zoltán Tüske, Michael Picheny

We propose a population-based Evolutionary Stochastic Gradient Descent (ESGD) framework for optimizing deep neural networks. ESGD combines SGD and gradient-free evolutionary algorithms as complementary algorithms in one …

Evolutionary AlgorithmsLanguage ModelingLanguage Modellingspeech-recognition+1

Uncertainty Sampling is Preconditioned Stochastic Gradient Descent on Zero-One Loss

2018-12-05 · NeurIPS 2018 12 · Stephen Mussmann, Percy Liang

Uncertainty sampling, a popular active learning algorithm, is used to reduce the amount of data required to learn a classifier, but it has been observed in practice to converge to different parameters depending on the in…

Active Learning