Peirce in the Machine: How Mixture of Experts Models Perform Hypothesis Construction
Mixture of experts is a prediction aggregation method in machine learning that aggregates the predictions of specialized experts. This method often outperforms Bayesian methods despite the Bayesian having stronger inductive guarantees. We argue that this is due to the greater functional capacity of mixture of experts. We prove that in a limiting case of mixture of experts will have greater capacity than equivalent Bayesian methods, which we vouchsafe through experiments on non-limiting cases. Finally, we conclude that mixture of experts is a type of abductive reasoning in the Peircian sense of hypothesis construction.
Code (1)
Tasks
Mixture-of-ExpertsSimilar Papers 제목 키워드 기반
On Parameter Estimation in Deviated Gaussian Mixture of Experts
We consider the parameter estimation problem in the deviated Gaussian mixture of experts in which the data are generated from $(1 - \lambda^{\ast}) g_0(Y| X)+ \lambda^{\ast} \sum_{i = 1}^{k_{\ast}} p_{i}^{\ast} f(Y|(a_{i…
Mixture-of-Expertsparameter estimationAbductive Inference and C. S. Peirce: 150 Years Later
This paper is about two things: (i) Charles Sanders Peirce (1837-1914) -- an iconoclastic philosopher and polymath who is among the greatest of American minds. (ii) Abductive inference -- a term coined by C. S. Peirce, w…
EconometricsHybrid Quantum-Classical Mixture of Experts: Unlocking Topological Advantage via Interference-Based Routing
The Mixture-of-Experts (MoE) architecture has emerged as a powerful paradigm for scaling deep learning models, yet it is fundamentally limited by challenges such as expert imbalance and the computational complexity of cl…
Quantum Machine LearningFederated LearningSeeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts
Multimodal Mixture-of-Experts (MoE) models have achieved remarkable performance on vision-language tasks. However, we identify a puzzling phenomenon termed Seeing but Not Thinking: models accurately perceive image conten…
Visual ReasoningWarping Peirce Quincuncial Panoramas
The Peirce quincuncial projection is a mapping of the surface of a sphere to the interior of a square. It is a conformal map except for four points on the equator. These points of non-conformality cause significant artif…