From Imitation to Prediction, Data Compression vs Recurrent Neural Networks for Natural Language Processing
In recent studies [1][13][12] Recurrent Neural Networks were used for generative processes and their surprising performance can be explained by their ability to create good predictions. In addition, data compression is also based on predictions. What the problem comes down to is whether a data compressor could be used to perform as well as recurrent neural networks in natural language processing tasks. If this is possible,then the problem comes down to determining if a compression algorithm is even more intelligent than a neural network in specific tasks related to human language. In our journey we discovered what we think is the fundamental difference between a Data Compression Algorithm and a Recurrent Neural Network.
Code (0)
등록된 구현이 없습니다.
Tasks
Data CompressionSimilar Papers 제목 키워드 기반
Image Compression predicated on Recurrent Iterated Function Systems
Recurrent iterated function systems (RIFSs) are improvements of iterated function systems (IFSs) using elements of the theory of Marcovian stochastic processes which can produce more natural looking images. We construct …
Image CompressionRestricted Recurrent Neural Networks
Recurrent Neural Network (RNN) and its variations such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), have become standard building blocks for learning online data of sequential nature in many research …
Language ModelingLanguage ModellingSyntactically Informed Text Compression with Recurrent Neural Networks
We present a self-contained system for constructing natural language models for use in text compression. Our system improves upon previous neural network based models by utilizing recent advances in syntactic parsing -- …
Text CompressionIt's a super deal -- train recurrent network on noisy data and get smooth prediction free
Recent research demonstrate that prediction of time series by predictive recurrent neural networks based on the noisy input generates a smooth anticipated trajectory. We examine influence of the noise component in both t…
PredictionTime SeriesTime Series AnalysisDeepZip: Lossless Data Compression using Recurrent Neural Networks
Sequential data is being generated at an unprecedented pace in various forms, including text and genomic data. This creates the need for efficient compression mechanisms to enable better storage, transmission and process…
Data Compression