Sequential Labeling with online Deep Learning
Deep learning has attracted great attention recently and yielded the state of the art performance in dimension reduction and classification problems. However, it cannot effectively handle the structured output prediction, e.g. sequential labeling. In this paper, we propose a deep learning structure, which can learn discriminative features for sequential labeling problems. More specifically, we add the inter-relationship between labels in our deep learning structure, in order to incorporate the context information from the sequential data. Thus, our model is more powerful than linear Conditional Random Fields (CRFs) because the objective function learns latent non-linear features so that target labeling can be better predicted. We pretrain the deep structure with stacked restricted Boltzmann machines (RBMs) for feature learning and optimize our objective function with online learning algorithm, a mixture of perceptron training and stochastic gradient descent. We test our model on different challenge tasks, and show that our model outperforms significantly over the completive baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningDimensionality ReductionSimilar Papers 제목 키워드 기반
SAC-GLAM: Improving Online RL for LLM agents with Soft Actor-Critic and Hindsight Relabeling
The past years have seen Large Language Models (LLMs) strive not only as generative models but also as agents solving textual sequential decision-making tasks. When facing complex environments where their zero-shot abili…
Decision MakingReinforcement Learning (RL)Sequential Decision MakingModeling sequential annotations for sequence labeling with crowds
Crowd sequential annotations can be an efficient and cost-effective way to build large datasets for sequence labeling. Different from tagging independent instances, for crowd sequential annotations the quality of label s…
validSequence Alignment Ensemble with a Single Neural Network for Sequence Labeling
Sequence labeling, in which a class or label is assigned to each token in a given input order, is a fundamental task in natural language processing. Many advanced neural network architectures have recently been proposed …
Part-Of-Speech TaggingPOSPOS TaggingA Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Sequential prediction problems such as imitation learning, where future observations depend on previous predictions (actions), violate the common i.i.d. assumptions made in statistical learning. This leads to poor perfor…
Imitation LearningStructured Prediction