Large-Scale Syntactic Language Modeling with Treelets
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMachine TranslationSpeech RecognitionSimilar Papers 제목 키워드 기반
Kernel Treelets
A new method for hierarchical clustering is presented. It combines treelets, a particular multiscale decomposition of data, with a projection on a reproducing kernel Hilbert space. The proposed approach, called kernel tr…
ClusteringTransformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scale
We introduce Transformer Grammars (TGs), a novel class of Transformer language models that combine (i) the expressive power, scalability, and strong performance of Transformers and (ii) recursive syntactic compositions, …
Inductive BiasLanguage ModelingLanguage ModellingSentenceModeling Global Syntactic Variation in English Using Dialect Classification
This paper evaluates global-scale dialect identification for 14 national varieties of English as a means for studying syntactic variation. The paper makes three main contributions: (i) introducing data-driven language ma…
ClassificationDialect IdentificationGeneral ClassificationBenchmarking Language Models for Code Syntax Understanding
Pre-trained language models have demonstrated impressive performance in both natural language processing and program understanding, which represent the input as a token sequence without explicitly modeling its structure.…
BenchmarkingGenerative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale
A syntactic language model (SLM) incrementally generates a sentence with its syntactic tree in a left-to-right manner. We present Generative Pretrained Structured Transformers (GPST), an unsupervised SLM at scale capable…
Constituency Grammar InductionLanguage ModelingLanguage ModellingNatural Language Inference+2