Statistical Foundation Behind Machine Learning and Its Impact on Computer Vision
This paper revisits the principle of uniform convergence in statistical learning, discusses how it acts as the foundation behind machine learning, and attempts to gain a better understanding of the essential problem that current deep learning algorithms are solving. Using computer vision as an example domain in machine learning, the discussion shows that recent research trends in leveraging increasingly large-scale data to perform pre-training for representation learning are largely to reduce the discrepancy between a practically tractable empirical loss and its ultimately desired but intractable expected loss. Furthermore, this paper suggests a few future research directions, predicts the continued increase of data, and argues that more fundamental research is needed on robustness, interpretability, and reasoning capabilities of machine learning by incorporating structure and knowledge.
Code (0)
등록된 구현이 없습니다.
Tasks
Representation LearningSimilar Papers 제목 키워드 기반
Why Machine Learning Cannot Ignore Maximum Likelihood Estimation
The growth of machine learning as a field has been accelerating with increasing interest and publications across fields, including statistics, but predominantly in computer science. How can we parse this vast literature …
BIG-bench Machine LearningNo Intelligence Without Statistics: The Invisible Backbone of Artificial Intelligence
The rapid ascent of artificial intelligence (AI) is often portrayed as a revolution born from computer science and engineering. This narrative, however, obscures a fundamental truth: the theoretical and methodological co…
Representation LearningDensity EstimationCausal Inference Tools for a Better Evaluation of Machine Learning
We present a comprehensive framework for applying rigorous statistical techniques from econometrics to analyze and improve machine learning systems. We introduce key statistical methods such as Ordinary Least Squares (OL…
Causal InferenceEconometricsFairnessregressionLag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting
Over the past years, foundation models have caused a paradigm shift in machine learning due to their unprecedented capabilities for zero-shot and few-shot generalization. However, despite the success of foundation models…
DecoderProbabilistic Time Series ForecastingTime SeriesTime Series Forecasting+1Towards Machine Wald
The past century has seen a steady increase in the need of estimating and predicting complex systems and making (possibly critical) decisions with limited information. Although computers have made possible the numerical …
Bayesian InferenceStochastic OptimizationUncertainty Quantification