A Short Information-Theoretic Analysis of Linear Auto-Regressive Learning
In this note, we give a short information-theoretic proof of the consistency of the Gaussian maximum likelihood estimator in linear auto-regressive models. Our proof yields nearly optimal non-asymptotic rates for parameter recovery and works without any invocation of stability in the case of finite hypothesis classes.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Edge of stability echo state networks
Echo State Networks (ESNs) are time-series processing models working under the Echo State Property (ESP) principle. The ESP is a notion of stability that imposes an asymptotic fading of the memory of the input. On the ot…
Estimating Linguistic Complexity for Science Texts
Evaluation of text difficulty is important both for downstream tasks like text simplification, and for supporting educators in classrooms. Existing work on automated text complexity analysis uses linear models with engin…
Feature EngineeringReading ComprehensionText SimplificationFrom Linearity to Non-Linearity: How Masked Autoencoders Capture Spatial Correlations
Masked Autoencoders (MAEs) have emerged as a powerful pretraining technique for vision foundation models. Despite their effectiveness, they require extensive hyperparameter tuning (masking ratio, patch size, encoder/deco…
It's Enough: Relaxing Diagonal Constraints in Linear Autoencoders for Recommendation
Linear autoencoder models learn an item-to-item weight matrix via convex optimization with L2 regularization and zero-diagonal constraints. Despite their simplicity, they have shown remarkable performance compared to sop…
DenoisingL2 RegularizationInformation Theoretic Analysis of DNN-HMM Acoustic Modeling
We propose an information theoretic framework for quantitative assessment of acoustic modeling for hidden Markov model (HMM) based automatic speech recognition (ASR). Acoustic modeling yields the probabilities of HMM sub…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech Recognition