Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
Selecting optimal parameters for a neural network architecture can often make the difference between mediocre and state-of-the-art performance. However, little is published which parameters and design choices should be evaluated or selected making the correct hyperparameter optimization often a "black art that requires expert experiences" (Snoek et al., 2012). In this paper, we evaluate the importance of different network design choices and hyperparameters for five common linguistic sequence tagging tasks (POS, Chunking, NER, Entity Recognition, and Event Detection). We evaluated over 50.000 different setups and found, that some parameters, like the pre-trained word embeddings or the last layer of the network, have a large impact on the performance, while other parameters, for example the number of LSTM layers or the number of recurrent units, are of minor importance. We give a recommendation on a configuration that performs well among different tasks.
Code (6)
Tasks
ChunkingEvent DetectionHyperparameter OptimizationNamed Entity Recognition (NER)POSWord EmbeddingsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling
Multi-task learning (MTL) has been studied recently for sequence labeling. Typically, auxiliary tasks are selected specifically in order to improve the performance of a target task. Jointly learning multiple tasks in a w…
ChunkingMulti-Task Learningnamed-entity-recognitionNamed Entity Recognition+2Supplementary Features of BiLSTM for Enhanced Sequence Labeling
Sequence labeling tasks require the computation of sentence representations for each word within a given sentence. A prevalent method incorporates a Bi-directional Long Short-Term Memory (BiLSTM) layer to enhance the seq…
Aspect-Based Sentiment AnalysisChinese Named Entity Recognitionnamed-entity-recognitionNamed Entity Recognition (NER)+6End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
State-of-the-art sequence labeling systems traditionally require large amounts of task-specific knowledge in the form of hand-crafted features and data pre-processing. In this paper, we introduce a novel neutral network …
Feature EngineeringNamed Entity RecognitionNamed Entity Recognition (NER)Part-Of-Speech Tagging+2End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF: A Reproducibility Study
We present a reproducibility study of the state-of-the-art neural architecture for sequence labeling proposed by Ma and Hovy (2016)\cite{ma2016end}. The original BiLSTM-CNN-CRF model combines character-level representati…
Structured PredictionLeveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling
Recurrent Neural Network (RNN) and one of its specific architectures, Long Short-Term Memory (LSTM), have been widely used for sequence labeling. In this paper, we first enhance LSTM-based sequence labeling to explicitly…
Natural Language UnderstandingSentenceslot-fillingSlot Filling