Learning Linear Block Codes with Gradient Quantization
This study investigates the problem of learning linear block codes optimized for Belief-Propagation decoders significantly improving performance compared to the state-of-the-art. Our previous research is extended with an enhanced system design that facilitates a more effective learning process for the parity check matrix. We simplify the input dataset, restrict the number of parameters to learn and improve the gradient back-propagation within the model. We also introduce novel optimizers specifically designed for discrete-valued weights. Based on conventional gradient computation, these optimizers provide discrete weights updates, enabling finer control and improving explainability of the learning process. Through these changes, we consistently achieve improved code performance, provided appropriately chosen hyper-parameters. To rigorously evaluate the performance of learned codes in the context of short to medium block lengths, we propose a comprehensive code performance assessment framework. This framework enables a fair comparison between our learning methodology and random search approaches, ensuring statistical significance in our results. The proposed model pave the way for a new approach to the efficient learning of linear block codes tailored to specific decoder structures.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderQuantizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving Quantization-aware Training of Low-Precision Network via Block Replacement on Full-Precision Counterpart
Quantization-aware training (QAT) is a common paradigm for network quantization, in which the training phase incorporates the simulation of the low-precision computation to optimize the quantization parameters in alignme…
Quantization8-bit Optimizers via Block-wise Quantization
Stateful optimizers maintain gradient statistics over time, e.g., the exponentially smoothed sum (SGD with momentum) or squared sum (Adam) of past gradient values. This state can be used to accelerate optimization compar…
Language ModelingLanguage ModellingMachine TranslationQuantizationLearning Linear Block Error Correction Codes
Error correction codes are a crucial part of the physical communication layer, ensuring the reliable transfer of data over noisy channels. The design of optimal linear block codes capable of being efficiently decoded is …
DecoderPush for Quantization: Deep Fisher Hashing
Current massive datasets demand light-weight access for analysis. Discrete hashing methods are thus beneficial because they map high-dimensional data to compact binary codes that are efficient to store and process, while…
QuantizationSemantic SimilaritySemantic Textual SimilarityBeyond Product Quantization: Deep Progressive Quantization for Image Retrieval
Product Quantization (PQ) has long been a mainstream for generating an exponentially large codebook at very low memory/time cost. Despite its success, PQ is still tricky for the decomposition of high-dimensional vector s…
Image RetrievalQuantizationRetrieval