Enhancing Pre-Trained Language Models with Sentence Position Embeddings for Rhetorical Roles Recognition in Legal Opinions
The legal domain is a vast and complex field that involves a considerable amount of text analysis, including laws, legal arguments, and legal opinions. Legal practitioners must analyze these texts to understand legal cases, research legal precedents, and prepare legal documents. The size of legal opinions continues to grow, making it increasingly challenging to develop a model that can accurately predict the rhetorical roles of legal opinions given their complexity and diversity. In this research paper, we propose a novel model architecture for automatically predicting rhetorical roles using pre-trained language models (PLMs) enhanced with knowledge of sentence position information within a document. Based on an annotated corpus from the LegalEval@SemEval2023 competition, we demonstrate that our approach requires fewer parameters, resulting in lower computational costs when compared to complex architectures employing a hierarchical model in a global-context, yet it achieves great performance. Moreover, we show that adding more attention to a hierarchical model based only on BERT in the local-context, along with incorporating sentence position information, enhances the results.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityPositionSentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Emu: Enhancing Multilingual Sentence Embeddings with Semantic Specialization
We present Emu, a system that semantically enhances multilingual sentence embeddings. Our framework fine-tunes pre-trained multilingual sentence embeddings using two main components: a semantic classifier and a language …
intent-classificationIntent ClassificationSemantic SimilaritySemantic Textual Similarity+4The Curious Case of Absolute Position Embeddings
Transformer language models encode the notion of word order using positional information. Most commonly, this positional information is represented by absolute position embeddings (APEs), that are learned from the pretra…
PositionSentenceSimple Techniques for Enhancing Sentence Embeddings in Generative Language Models
Sentence Embedding stands as a fundamental task within the realm of Natural Language Processing, finding extensive application in search engines, expert systems, and question-and-answer platforms. With the continuous evo…
Prompt EngineeringSentenceSentence EmbeddingSentence-Embedding+1SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features
Models based on large-pretrained language models, such as S(entence)BERT, provide effective and efficient sentence embeddings that show high correlation to human similarity ratings, but lack interpretability. On the othe…
Abstract Meaning RepresentationNegationSentenceSentence Embeddings+1Frustratingly Easy Performance Improvements for Low-resource Setups: A Tale on BERT and Segment Embeddings
As input representation for each sub-word, the original BERT architecture proposes the sum of the sub-word embedding, position embedding and a segment embedding. Sub-word and position embeddings are well-known and studie…
Dependency ParsingPositionSentence