Differentiable Time-Varying Linear Prediction in the Context of End-to-End Analysis-by-Synthesis
Training the linear prediction (LP) operator end-to-end for audio synthesis in modern deep learning frameworks is slow due to its recursive formulation. In addition, frame-wise approximation as an acceleration method cannot generalise well to test time conditions where the LP is computed sample-wise. Efficient differentiable sample-wise LP for end-to-end training is the key to removing this barrier. We generalise the efficient time-invariant LP implementation from the GOLF vocoder to time-varying cases. Combining this with the classic source-filter model, we show that the improved GOLF learns LP coefficients and reconstructs the voice better than its frame-wise counterparts. Moreover, in our listening test, synthesised outputs from GOLF scored higher in quality ratings than the state-of-the-art differentiable WORLD vocoder.
Code (4)
Tasks
Audio SynthesisSimilar Papers 제목 키워드 기반
Differentiable Grey-box Modelling of Phaser Effects using Frame-based Spectral Processing
Machine learning approaches to modelling analog audio effects have seen intensive investigation in recent years, particularly in the context of non-linear time-invariant effects such as guitar amplifiers. For modulation …
Probabilistic sequential matrix factorization
We introduce the probabilistic sequential matrix factorization (PSMF) method for factorizing time-varying and non-stationary datasets consisting of high-dimensional time-series. In particular, we consider nonlinear Gauss…
ImputationMultivariate Time Series ForecastingMultivariate Time Series ImputationState Space Models+3Deluca -- A Differentiable Control Library: Environments, Methods, and Benchmarking
We present an open-source library of natively differentiable physics and robotics environments, accompanied by gradient-based control methods and a benchmark-ing suite. The introduced environments allow auto-differentiat…
BenchmarkingOpenAI GymOnline Proximal-ADMM For Time-varying Constrained Convex Optimization
This paper considers a convex optimization problem with cost and constraints that evolve over time. The function to be minimized is strongly convex and possibly non-differentiable, and variables are coupled through linea…
Learning Constrained Adaptive Differentiable Predictive Control Policies With Guarantees
We present differentiable predictive control (DPC), a method for learning constrained neural control policies for linear systems with probabilistic performance guarantees. We employ automatic differentiation to obtain di…
Continuous ControlImitation LearningModel Predictive Control