Evaluating the Effectiveness of Efficient Neural Architecture Search for Sentence-Pair Tasks
Neural Architecture Search (NAS) methods, which automatically learn entire neural model or individual neural cell architectures, have recently achieved competitive or state-of-the-art (SOTA) performance on variety of natural language processing and computer vision tasks, including language modeling, natural language inference, and image classification. In this work, we explore the applicability of a SOTA NAS algorithm, Efficient Neural Architecture Search (ENAS) (Pham et al., 2018) to two sentence pair tasks, paraphrase detection and semantic textual similarity. We use ENAS to perform a micro-level search and learn a task-optimized RNN cell architecture as a drop-in replacement for an LSTM. We explore the effectiveness of ENAS through experiments on three datasets (MRPC, SICK, STS-B), with two different models (ESIM, BiLSTM-Max), and two sets of embeddings (Glove, BERT). In contrast to prior work applying ENAS to NLP tasks, our results are mixed -- we find that ENAS architectures sometimes, but not always, outperform LSTMs and perform similarly to random architecture search.
Code (0)
등록된 구현이 없습니다.
Tasks
image-classificationImage ClassificationLanguage ModelingLanguage ModellingMRPCNatural Language InferenceNeural Architecture SearchSemantic Textual SimilaritySentenceSTSSTS-BMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Sentence Pair Embeddings Based Evaluation Metric for Abstractive and Extractive Summarization
The development of an automatic evaluation metric remains an open problem in text generation. Widely used evaluation metrics, like ROUGE and BLEU, are based on exact word matching and fail to capture semantic similarity.…
Extractive SummarizationNatural Language InferenceSemantic SimilaritySemantic Textual Similarity+5Training and Evaluating Multimodal Word Embeddings with Large-scale Web Annotated Images
In this paper, we focus on training and evaluating effective word embeddings with both text and visual information. More specifically, we introduce a large-scale dataset with 300 million sentences describing over 40 mill…
Image RetrievalSentenceWord EmbeddingsReadability-based Sentence Ranking for Evaluating Text Simplification
We propose a new method for evaluating the readability of simplified sentences through pair-wise ranking. The validity of the method is established through in-corpus and cross-corpus evaluation experiments. The approach …
ArticlesCross-corpusSentenceText SimplificationPointwise Paraphrase Appraisal is Potentially Problematic
The prevailing approach for training and evaluating paraphrase identification models is constructed as a binary classification problem: the model is given a pair of sentences, and is judged by how accurately it classifie…
Binary ClassificationParaphrase IdentificationDialogueCSE: Dialogue-based Contrastive Learning of Sentence Embeddings
Learning sentence embeddings from dialogues has drawn increasing attention due to its low annotation cost and high domain adaptability. Conventional approaches employ the siamese-network for this task, which obtains the …
Contrastive LearningSemantic Textual SimilaritySentenceSentence Embeddings