Byte-Pair Encoding for Text-to-SQL Generation

2019-10-20 · Samuel Müller, Andreas Vlachos

Neural sequence-to-sequence models provide a competitive approach to the task of mapping a question in natural language to an SQL query, also referred to as text-to-SQL generation. The Byte-Pair Encoding algorithm (BPE) has previously been used to improve machine translation (MT) between natural languages. In this work, we adapt BPE for text-to-SQL generation. As the datasets for this task are rather small compared to MT, we present a novel stopping criterion that prevents overfitting the BPE encoding to the training set. Additionally, we present AST BPE, which is a version of BPE that uses the Abstract Syntax Tree (AST) of the SQL statement to guide BPE merges and therefore produce BPE encodings that generalize better. We improved the accuracy of a strong attentive seq2seq baseline on five out of six English text-to-SQL tasks while reducing training time by more than 50% on four of them due to the shortened targets. Finally, on two of these tasks we exceeded previously reported accuracies.

📄 PDF Abstract BibTeX arXiv:1910.08962

Code (1)

SamuelGabriel/sqlbpe 공식 구현 pytorch

Tasks

Machine Translation Text to SQL Text-To-SQL Translation

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

Seq2Seq Seq2Seq, or Sequence To Sequence, is a model used in sequence prediction tasks, such as language modelling and machine translation. The idea is to use one…

Multidimensional Byte Pair Encoding: Shortened Sequences for Improved Visual Data Generation

2024-11-15 · Tim Elsner, Paula Usinger, Julius Nehring-Wirxel, Gregor Kobsik 외

In language processing, transformers benefit greatly from text being condensed. This is achieved through a larger vocabulary that captures word fragments instead of plain characters. This is often done with Byte Pair Enc…

Composing Byte-Pair Encodings for Morphological Sequence Classification

2020-12-01 · UDW (COLING) 2020 12 · Adam Ek, Jean-Philippe Bernardy

Byte-pair encodings is a method for splitting a word into sub-word tokens, a language model then assigns contextual representations separately to each of these tokens. In this paper, we evaluate four different methods of…

Classification Language Modeling Language Modelling

When Molecular GAN Meets Byte-Pair Encoding

2024-09-29 · Huidong Tang, Chen Li, Yasuhiko Morimoto

Deep generative models, such as generative adversarial networks (GANs), are pivotal in discovering novel drug-like candidates via de novo molecular generation. However, traditional character-wise tokenizers often struggl…

Computational Efficiency Diversity

Code Completion using Neural Attention and Byte Pair Encoding

2020-04-14 · Youri Arkesteijn, Nikhil Saldanha, Bastijn Kostense

In this paper, we aim to do code completion based on implementing a Neural Network from Li et. al.. Our contribution is that we use an encoding that is in-between character and word encoding called Byte Pair Encoding (BP…

Code Completion

Improving Natural Language Understanding by Reverse Mapping Bytepair Encoding

2019-11-01 · CONLL 2019 11 · Chaodong Tong, Huailiang Peng, Qiong Dai, Lei Jiang 외

We propose a method called reverse mapping bytepair encoding, which maps named-entity information and other word-level linguistic features back to subwords during the encoding procedure of bytepair encoding (BPE). We emp…

Natural Language Understanding RTE SST-2