Exploiting LLMs' Reasoning Capability to Infer Implicit Concepts in Legal Information Retrieval
Statutory law retrieval is a typical problem in legal language processing, that has various practical applications in law engineering. Modern deep learning-based retrieval methods have achieved significant results for this problem. However, retrieval systems relying on semantic and lexical correlations often exhibit limitations, particularly when handling queries that involve real-life scenarios, or use the vocabulary that is not specific to the legal domain. In this work, we focus on overcoming this weaknesses by utilizing the logical reasoning capabilities of large language models (LLMs) to identify relevant legal terms and facts related to the situation mentioned in the query. The proposed retrieval system integrates additional information from the term--based expansion and query reformulation to improve the retrieval accuracy. The experiments on COLIEE 2022 and COLIEE 2023 datasets show that extra knowledge from LLMs helps to improve the retrieval result of both lexical and semantic ranking models. The final ensemble retrieval system outperformed the highest results among all participating teams in the COLIEE 2022 and 2023 competitions.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalLogical ReasoningRetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DetectBench: Can Large Language Model Detect and Piece Together Implicit Evidence?
Detecting evidence within the context is a key step in the process of reasoning task. Evaluating and enhancing the capabilities of LLMs in evidence detection will strengthen context-based reasoning performance. This pape…
Language ModelingLanguage ModellingLarge Language ModelMultiple-choiceGAPMAP: Mapping Scientific Knowledge Gaps in Biomedical Literature Using Large Language Models
Scientific progress is driven by the deliberate articulation of what remains unknown. This study investigates the ability of large language models (LLMs) to identify research knowledge gaps in the biomedical literature. …
Domain AdaptationSelf-Training Elicits Concise Reasoning in Large Language Models
Chain-of-thought (CoT) reasoning has enabled large language models (LLMs) to utilize additional computation through intermediate tokens to solve complex tasks. However, we posit that typical reasoning traces contain many…
GSM8KIn-Context LearningMathWhere Norms and References Collide: Evaluating LLMs on Normative Reasoning
Embodied agents, such as robots, will need to interact in situated environments where successful communication often depends on reasoning over social norms: shared expectations that constrain what actions are appropriate…
Inference-Time Reasoning Selectively Reduces Implicit Social Bias in Large Language Models
Drawing on constructs from psychology, prior work has identified a distinction between explicit and implicit bias in large language models (LLMs). While many LLMs undergo post-training alignment and safety procedures to …