A Technical Note on the Architectural Effects on Maximum Dependency Lengths of Recurrent Neural Networks
This work proposes a methodology for determining the maximum dependency length of a recurrent neural network (RNN), and then studies the effects of architectural changes, including the number and neuron count of layers, on the maximum dependency lengths of traditional RNN, gated recurrent unit (GRU), and long-short term memory (LSTM) models.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Polyphonic Piano Transcription Using Autoregressive Multi-State Note Model
Recent advances in polyphonic piano transcription have been made primarily by a deliberate design of neural network architectures that detect different note states such as onset or sustain and model the temporal evolutio…
Notes on Generalizing the Maximum Entropy Principle to Uncertain Data
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We se…
Brief technical note on linearizing recurrent neural networks (RNNs) before vs after the pointwise nonlinearity
Linearization of the dynamics of recurrent neural networks (RNNs) is often used to study their properties. The same RNN dynamics can be written in terms of the ``activations" (the net inputs to each unit, before its poin…
The IULA Spanish LSP Treebank
This paper presents the IULA Spanish LSP Treebank, a dependency treebank of over 41,000 sentences of different domains (Law, Economy, Computing Science, Environment, and Medicine), developed in the framework of the Europ…
Technical notes: Syntax-aware Representation Learning With Pointer Networks
This is a work-in-progress report, which aims to share preliminary results of a novel sequence-to-sequence schema for dependency parsing that relies on a combination of a BiLSTM and two Pointer Networks (Vinyals et al., …
Dependency ParsingregressionRepresentation Learning