Efficient Word Lattice Generation for Joint Word Segmentation and POS Tagging in Japanese

Reduce Meaningless Words for Joint Chinese Word Segmentation and Part-of-speech Tagging

Conventional statistics-based methods for joint Chinese word segmentation and part-of-speech tagging (S&T) have generalization ability to recognize new words that do not appear in the training data. An undesirable side e…

A Lattice-based Framework for Joint Chinese Word Segmentation, POS Tagging and Parsing

A Graph-based Lattice Dependency Parser for Joint Morphological Segmentation and Syntactic Analysis

Space-delimited words in Turkish and Hebrew text can be further segmented into meaningful units, but syntactic and semantic context is necessary to predict segmentation. At the same time, predicting correct syntactic str…

Lattice-Based Transformer Encoder for Neural Machine Translation

Neural machine translation (NMT) takes deterministic sequences for source representations. However, either word-level or subword-level segmentations have multiple choices to split a source sequence with different word se…

LATTE: Lattice ATTentive Encoding for Character-based Word Segmentation

A character sequence comprises at least one or more segmentation alternatives. This can be considered segmentation ambiguity and may weaken segmentation performance in word segmentation. Proper handling of such ambiguity…