Two Ridge Solutions for the Incremental Broad Learning System on Added Nodes
The original Broad Learning System (BLS) on new added nodes and its existing efficient implementation both assume the ridge parameter lambda -> 0 in the ridge inverse to approximate the generalized inverse, and compute the generalized inverse solution for the output weights. In this paper, we propose two ridge solutions for the output weights in the BLS on added nodes, where lambda -> 0 is no longer assumed, and lambda can be any positive real number. One of the proposed ridge solutions computes the output weights from the inverse Cholesky factor, which is updated efficiently by extending the existing inverse Cholesky factorization. The other proposed ridge solution computes the output weights from the ridge inverse, and updates the ridge inverse by extending the Greville's method that is a classical tool to compute the generalized inverse of partitioned matrices. For the proposed efficient ridge solution based on the inverse Cholesky factor, we also develop another implementation that is numerically more stable when the ridge parameter lambda is very small. The proposed ridge solution based on the ridge inverse and the numerically more stable implementation of the proposed efficient ridge solution require the same complexity as the original BLS and the existing efficient BLS, respectively. Moreover, the speedups of the proposed efficient ridge solution to the original BLS and the existing efficient BLS are 3 and more than 1.67 respectively, when the computational complexities for each update are compared, and the speedups are 1.99 - 2.52 and 1.31 - 1.58, respectively, when the total training time is compared by numerical experiments. On the other hand, our numerical experiments show that both the proposed ridge solutions for BLS achieve better testing accuracies than the original BLS and the existing efficient BLS.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Low-Memory Implementations of Ridge Solutions for Broad Learning System with Incremental Learning
The existing low-memory BLS implementation proposed recently avoids the need for storing and inverting large matrices, to achieve efficient usage of memories. However, the existing low-memory BLS implementation sacrifice…
Incremental LearningEfficient Decremental Learning Algorithms for Broad Learning System
The decremented learning algorithms are required in machine learning, to prune redundant nodes and remove obsolete inline training samples. In this paper, an efficient decremented learning algorithm to prune redundant no…
Incremental LearningTwo Efficient Ridge Solutions for the Incremental Broad Learning System on Added Inputs
This paper proposes the recursive and square-root BLS algorithms to improve the original BLS for new added inputs, which utilize the inverse and inverse Cholesky factor of the Hermitian matrix in the ridge inverse, respe…
LEMMAReducing the Computational Complexity of Pseudoinverse for the Incremental Broad Learning System on Added Inputs
In this brief, we improve the Broad Learning System (BLS) [7] by reducing the computational complexity of the incremental learning for added inputs. We utilize the inverse of a sum of matrices in [8] to improve a step in…
Incremental LearningWhen More Data Hurts: A Troubling Quirk in Developing Broad-Coverage Natural Language Understanding Systems
In natural language understanding (NLU) production systems, users' evolving needs necessitate the addition of new features over time, indexed by new symbols added to the meaning representation space. This requires additi…
Intent RecognitionNatural Language UnderstandingSemantic Parsing