Convolutions Are All You Need (For Classifying Character Sequences)
While recurrent neural networks (RNNs) are widely used for text classification, they demonstrate poor performance and slow convergence when trained on long sequences. When text is modeled as characters instead of words, the longer sequences make RNNs a poor choice. Convolutional neural networks (CNNs), although somewhat less ubiquitous than RNNs, have an internal structure more appropriate for long-distance character dependencies. To better understand how CNNs and RNNs differ in handling long sequences, we use them for text classification tasks in several character-level social media datasets. The CNN models vastly outperform the RNN models in our experiments, suggesting that CNNs are superior to RNNs at learning to classify character-level data.
Code (0)
등록된 구현이 없습니다.
Tasks
AllClassificationDocument ClassificationGeneral ClassificationLanguage ModelingLanguage ModellingMachine Translationtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Modulation Pattern Detection Using Complex Convolutions in Deep Learning
Transceivers used for telecommunications transmit and receive specific modulation patterns that are represented as sequences of complex numbers. Classifying modulation patterns is challenging because noise and channel im…
Deep LearningEnabling Long FFT Convolutions on Memory-Constrained FPGAs via Chunking
The need for long-context reasoning has led to alternative neural network architectures besides Transformers and self-attention, a popular model being Hyena, which employs causal 1D-convolutions implemented with FFTs. Lo…
Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences
To mitigate the computational complexity in the self-attention mechanism on long sequences, linear attention utilizes computation tricks to achieve linear complexity, while state space models (SSMs) popularize a favorabl…
Language ModelingLanguage ModellingLong-range modelingState Space ModelsUsing Signal Processing in Tandem With Adapted Mixture Models for Classifying Genomic Signals
Genomic signal processing has been used successfully in bioinformatics to analyze biomolecular sequences and gain varied insights into DNA structure, gene organization, protein binding, sequence evolution, etc. But chall…
Sequence-based Dynamic Handwriting Analysis for Parkinson's Disease Detection with One-dimensional Convolutions and BiGRUs
Parkinson's disease (PD) is commonly characterized by several motor symptoms, such as bradykinesia, akinesia, rigidity, and tremor. The analysis of patients' fine motor control, particularly handwriting, is a powerful to…
ClassificationGeneral Classification