Learning Moderately Input-Sensitive Functions: A Case Study in QR Code Decoding
The hardness of learning a function that attains a target task relates to its input-sensitivity. For example, image classification tasks are input-insensitive as minor corruptions should not affect the classification results, whereas arithmetic and symbolic computation, which have been recently attracting interest, are highly input-sensitive as each input variable connects to the computation results. This study presents the first learning-based Quick Response (QR) code decoding and investigates learning functions of medium sensitivity. Our experiments reveal that Transformers can successfully decode QR codes, even beyond the theoretical error-correction limit, by learning the structure of embedded texts. They generalize from English-rich training data to other languages and even random strings. Moreover, we observe that the Transformer-based QR decoder focuses on data bits while ignoring error-correction bits, suggesting a decoding mechanism distinct from standard QR code readers.
Code (0)
등록된 구현이 없습니다.
Tasks
Decoderimage-classificationImage ClassificationSensitivitySimilar Papers 제목 키워드 기반
Representation of linguistic form and function in recurrent neural networks
We present novel methods for analyzing the activation patterns of RNNs from a linguistic point of view and explore the types of linguistic structure they learn. As a case study, we use a multi-task gated recurrent networ…
FormLanguage ModelingLanguage ModellingSentence+1Large Population Sizes and Crossover Help in Dynamic Environments
Dynamic linear functions on the hypercube are functions which assign to each bit a positive weight, but the weights change over time. Throughout optimization, these functions maintain the same global optimum, and never h…
Poly-NL: Linear Complexity Non-local Layers with Polynomials
Spatial self-attention layers, in the form of Non-Local blocks, introduce long-range dependencies in Convolutional Neural Networks by computing pairwise similarities among all possible positions. Such pairwise functions …
Face DetectionInstance SegmentationSemantic SegmentationPoly-NL: Linear Complexity Non-Local Layers With 3rd Order Polynomials
Spatial self-attention layers, in the form of Non-Local blocks, introduce long-range dependencies in Convolutional Neural Networks by computing pairwise similarities among all possible positions. Such pairwise functi…
Face DetectionInstance SegmentationSemantic SegmentationUnderstanding the Parameter Space Geometry of Transformers Encoding Boolean Functions
Transformers consistently fail to learn certain simple functions that are provably expressible with specific parameter settings. This gap between learnability and expressivity is particularly prominent for sensitive func…