Tensor-variate Mixture of Experts for Proportional Myographic Control of a Robotic Hand
When data are organized in matrices or arrays of higher dimensions (tensors), classical regression methods first transform these data into vectors, therefore ignoring the underlying structure of the data and increasing the dimensionality of the problem. This flattening operation typically leads to overfitting when only few training data is available. In this paper, we present a mixture-of-experts model that exploits tensorial representations for regression of tensor-valued data. The proposed formulation takes into account the underlying structure of the data and remains efficient when few training data are available. Evaluation on artificially generated data, as well as offline and real-time experiments recognizing hand movements from tactile myography prove the effectiveness of the proposed approach.
Code (1)
Tasks
Mixture-of-ExpertsregressionSimilar Papers 제목 키워드 기반
Nonlinear Semi-Parametric Models for Survival Analysis
Semi-parametric survival analysis methods like the Cox Proportional Hazards (CPH) regression (Cox, 1972) are a popular approach for survival analysis. These methods involve fitting of the log-proportional hazard as a fun…
regressionSurvival AnalysisTenExp: Mixture-of-Experts-Based Tensor Decomposition Structure Search Framework
Recently, tensor decompositions continue to emerge and receive increasing attention. Selecting a suitable tensor decomposition to exactly capture the low-rank structures behind the data is at the heart of the tensor deco…
DualSparse-MoE: Coordinating Tensor/Neuron-Level Sparsity with Expert Partition and Reconstruction
Mixture of Experts (MoE) has become a mainstream architecture for building Large Language Models (LLMs) by reducing per-token computation while enabling model scaling. It can be viewed as partitioning a large Feed-Forwar…
Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Models
Recently, Mixture-of-Experts (short as MoE) architecture has achieved remarkable success in increasing the model capacity of large-scale language models. However, MoE requires incorporating significantly more parameters …
Language ModelingLanguage ModellingMixture-of-ExpertsMulti-Task Learning+2Elliptically-Contoured Tensor-variate Distributions with Application to Improved Image Learning
Statistical analysis of tensor-valued data has largely used the tensor-variate normal (TVN) distribution that may be inadequate when data comes from distributions with heavier or lighter tails. We study a general family …