On the Effect of Quantization on Dynamic Mode Decomposition
Dynamic Mode Decomposition (DMD) is a widely used data-driven algorithm for estimating the Koopman Operator.This paper investigates how the estimation process is affected when the data is quantized. Specifically, we examine the fundamental connection between estimates of the operator obtained from unquantized data and those from quantized data. Furthermore, using the law of large numbers, we demonstrate that, under a large data regime, the quantized estimate can be considered a regularized version of the unquantized estimate. This key theoretical finding paves the way to accurately recover the unquantized estimate from quantized data. We also explore the relationship between the two estimates in the finite data regime. The theory is validated through repeated numerical experiments conducted on three different dynamical systems.
Code (0)
등록된 구현이 없습니다.
Tasks
QuantizationSimilar Papers 제목 키워드 기반
Optimized Product Quantization for Approximate Nearest Neighbor Search
Product quantization is an effective vector quantization approach to compactly encode high-dimensional vectors for fast approximate nearest neighbor (ANN) search. The essence of product quantization is to decompose the o…
QuantizationQUAD: Quantization and Parameter-Efficient Tuning of LLM with Activation Decomposition
Large Language Models (LLMs) excel in diverse applications but suffer inefficiency due to massive scale. While quantization reduces computational costs, existing methods degrade accuracy in medium-sized LLMs (e.g., Llama…
parameter-efficient fine-tuningQuantizationMulti-Scale Dequant: Eliminating Dequantization Bottleneck via Activation Decomposition for Efficient LLM Inference
Quantization is essential for efficient large language model (LLM) inference, yet the dequantization step-converting low-bit weights back to high-precision for matrix multiplication has become a critical bottleneck on mo…
Local Orthogonal Decomposition for Maximum Inner Product Search
Inverted file and asymmetric distance computation (IVFADC) have been successfully applied to approximate nearest neighbor search and subsequently maximum inner product search. In such a framework, vector quantization is …
QuantizationUnlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression
Key-value~(KV) caching is an important technique to accelerate the inference of large language models~(LLMs), but incurs significant memory overhead. To compress the size of KV cache, existing methods often compromise pr…
QuantizationTensor Decomposition