Beyond Paragraphs: NLP for Long Sequences
In this tutorial, we aim at bringing interested NLP researchers up to speed about the recent and ongoing techniques for document-level representation learning. Additionally, our goal is to reveal new research opportunities to the audience, which will hopefully bring us closer to address existing challenges in this domain.
Code (1)
Tasks
Representation LearningSimilar Papers 제목 키워드 기반
Scaling State-Space Models from Lines to Paragraphs: An Ablation of Mamba-based OCR
End-to-end OCR increasingly relies on autoregressive sequence models, where the quadratic cost of Transformer attention limits efficient transcription of long, paragraph-level text. State-Space Models (SSMs) such as Mamb…
Generating Wikipedia by Summarizing Long Sequences
We show that generating English Wikipedia articles can be approached as a multi- document summarization of source documents. We use extractive summarization to coarsely identify salient information and a neural abstracti…
ArticlesDecoderDocument SummarizationExtractive Summarization+2Human Genome Book: Words, Sentences and Paragraphs
Since the completion of the human genome sequencing project in 2001, significant progress has been made in areas such as gene regulation editing and protein structure prediction. However, given the vast amount of genomic…
Protein Structure PredictionSentence segmentationTransfer LearningWhen an Image Tells a Story: The Role of Visual and Semantic Information for Generating Paragraph Descriptions
Generating multi-sentence image descriptions is a challenging task, which requires a good model to produce coherent and accurate paragraphs, describing salient objects in the image. We argue that multiple sources of info…
Image Paragraph CaptioningSentenceRanking and Retrieval of Image Sequences From Multiple Paragraph Queries
We propose a method to rank and retrieve image sequences from a natural language text query, consisting of multiple sentences or paragraphs. One of the method's key applications is to visualize visitors' text-only review…
RetrievalSentence