An Evaluation Framework for Legal Document Summarization
A law practitioner has to go through numerous lengthy legal case proceedings for their practices of various categories, such as land dispute, corruption, etc. Hence, it is important to summarize these documents, and ensure that summaries contain phrases with intent matching the category of the case. To the best of our knowledge, there is no evaluation metric that evaluates a summary based on its intent. We propose an automated intent-based summarization metric, which shows a better agreement with human evaluation as compared to other automated metrics like BLEU, ROUGE-L etc. in terms of human satisfaction. We also curate a dataset by annotating intent phrases in legal documents, and show a proof of concept as to how this system can be automated. Additionally, all the code and data to generate reproducible results is available on Github.
Code (1)
Tasks
Document SummarizationLegal Document SummarizationSimilar Papers 제목 키워드 기반
Legal Case Document Summarization: Extractive and Abstractive Methods and their Evaluation
Summarization of legal case judgement documents is a challenging problem in Legal NLP. However, not much analyses exist on how different families of summarization models (e.g., extractive vs. abstractive) perform when ap…
Abstractive Text SummarizationDocument SummarizationOpen-Ended Question AnsweringAugAbEx: Bridging Abstractive and Extractive Legal Summarization
Automatic summarization of legal judgments liberates law professionals from heavy cognitive burden due to the complexity of the language, context-sensitive legal jargon, and the length of the document. Caveats of abstrac…
PersonaMatrix: A Recipe for Persona-Aware Evaluation of Legal Summarization
Legal documents are often long, dense, and difficult to comprehend, not only for laypeople but also for legal experts. While automated document summarization has great potential to improve access to legal knowledge, prev…
Document SummarizationAdvantages of Domain Knowledge Injection for Legal Document Summarization: A Case Study on Summarizing Indian Court Judgments in English and Hindi
Summarizing Indian legal court judgments is a complex task not only due to the intricate language and unstructured nature of the legal texts, but also since a large section of the Indian population does not understand th…
Document SummarizationReGal: A First Look at PPO-based Legal AI for Judgment Prediction and Summarization in India
This paper presents an early exploration of reinforcement learning methodologies for legal AI in the Indian context. We introduce Reinforcement Learning-based Legal Reasoning (ReGal), a framework that integrates Multi-Ta…
Document SummarizationReinforcement LearningLegal Reasoning