paper-with-me

홈 › Papers

Adaptive Error-Bounded Hierarchical Matrices for Efficient Neural Network Compression

2024-09-11 · John Mango, Ronald Katende

This paper introduces a dynamic, error-bounded hierarchical matrix (H-matrix) compression method tailored for Physics-Informed Neural Networks (PINNs). The proposed approach reduces the computational complexity and memory demands of large-scale physics-based models while preserving the essential properties of the Neural Tangent Kernel (NTK). By adaptively refining hierarchical matrix approximations based on local error estimates, our method ensures efficient training and robust model performance. Empirical results demonstrate that this technique outperforms traditional compression methods, such as Singular Value Decomposition (SVD), pruning, and quantization, by maintaining high accuracy and improving generalization capabilities. Additionally, the dynamic H-matrix method enhances inference speed, making it suitable for real-time applications. This approach offers a scalable and efficient solution for deploying PINNs in complex scientific and engineering domains, bridging the gap between computational feasibility and real-world applicability.

📄 PDF Abstract BibTeX arXiv:2409.07028

Code (0)

등록된 구현이 없습니다.

Tasks

Efficient Neural NetworkNeural Network CompressionQuantization

Similar Papers 제목 키워드 기반

SRN-SZ: Deep Leaning-Based Scientific Error-bounded Lossy Compression with Super-resolution Neural Networks

2023-09-07 · Jinyang Liu, Sheng Di, Sian Jin, Kai Zhao 외

The fast growth of computational power and scales of modern super-computing systems have raised great challenges for the management of exascale scientific data. To maintain the usability of scientific data, error-bound l…

Super-Resolution

Accelerating Communication in Deep Learning Recommendation Model Training with Dual-Level Adaptive Lossy Compression

2024-07-05 · Hao Feng, Boyuan Zhang, Fanjiang Ye, Min Si 외

DLRM is a state-of-the-art recommendation system model that has gained widespread adoption across various industry applications. The large size of DLRM models, however, necessitates the use of multiple devices/GPUs for e…

COMET: A Novel Memory-Efficient Deep Learning Training Framework by Using Error-Bounded Lossy Compression

2021-11-18 · Sian Jin, Chengming Zhang, Xintong Jiang, Yunhe Feng 외

Training wide and deep neural networks (DNNs) require large amounts of storage resources such as memory because the intermediate activation data must be saved in the memory during forward propagation and then restored fo…

Data Compression

Exploring Autoencoder-based Error-bounded Compression for Scientific Data

2021-05-25 · Jinyang Liu, Sheng Di, Kai Zhao, Sian Jin 외

Error-bounded lossy compression is becoming an indispensable technique for the success of today's scientific projects with vast volumes of data produced during simulations or instrument data acquisitions. Not only can it…

Image Compression

Random Paraunitary Projections

2021-06-20 · Ricardo L. De Queiroz

Transforms using random matrices have been found to have many applications. We are concerned with the projection of a signal onto Gaussian-distributed random orthogonal bases. We also would like to easily invert the proc…