LLMs & Legal Aid: Understanding Legal Needs Exhibited Through User Queries

2025-01-03 · Michal Kuk, Jakub Harasta

The paper presents a preliminary analysis of an experiment conducted by Frank Bold, a Czech expert group, to explore user interactions with GPT-4 for addressing legal queries. Between May 3, 2023, and July 25, 2023, 1,252 users submitted 3,847 queries. Unlike studies that primarily focus on the accuracy, factuality, or hallucination tendencies of large language models (LLMs), our analysis focuses on the user query dimension of the interaction. Using GPT-4o for zero-shot classification, we categorized queries on (1) whether users provided factual information about their issue (29.95%) or not (70.05%), (2) whether they sought legal information (64.93%) or advice on the course of action (35.07\%), and (3) whether they imposed requirements to shape or control the model's answer (28.57%) or not (71.43%). We provide both quantitative and qualitative insight into user needs and contribute to a better understanding of user engagement with LLMs.

📄 PDF Abstract BibTeX arXiv:2501.01711

Code (0)

등록된 구현이 없습니다.

Tasks

Hallucinationzero-shot-classificationZero-Shot Learning

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Adam 설명 없음

Residual Connection 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Multi-Head Attention 설명 없음

ASVRI-Legal: Fine-Tuning LLMs with Retrieval Augmented Generation for Enhanced Legal Regulation

2025-11-05 · One Octadion, Bondan Sapta Prakoso, Nanang Yudi Setiawan, Novanto Yudistira arxiv

In this study, we explore the fine-tuning of Large Language Models (LLMs) to better support policymakers in their crucial work of understanding, analyzing, and crafting legal regulations. To equip the model with a deep u…

LeKUBE: A Legal Knowledge Update BEnchmark

2024-07-19 · Changyue Wang, Weihang Su, Hu Yiran, Qingyao Ai 외

Recent advances in Large Language Models (LLMs) have significantly shaped the applications of AI in multiple fields, including the studies of legal intelligence. Trained on extensive legal texts, including statutes and l…

Legal Reasoning

ALKAFI-LLAMA3: Fine-Tuning LLMs for Precise Legal Understanding in Palestine

2024-12-19 · Rabee Qasem, Mohannad Hendi, Banan Tantour

Large Language Models (LLMs) have demonstrated remarkable potential in diverse domains, yet their application in the legal sector, particularly in low-resource contexts, remains limited. This study addresses the challeng…

ELLA: Empowering LLMs for Interpretable, Accurate and Informative Legal Advice

2024-08-13 · Yutong Hu, Kangcheng Luo, Yansong Feng

Despite remarkable performance in legal consultation exhibited by legal Large Language Models(LLMs) combined with legal article retrieval components, there are still cases when the advice given is incorrect or baseless. …

Articles

LawBench: Benchmarking Legal Knowledge of Large Language Models

2023-09-28 · Zhiwei Fei, Xiaoyu Shen, Dawei Zhu, Fengzhe Zhou 외

Large language models (LLMs) have demonstrated strong capabilities in various aspects. However, when applying them to the highly specialized, safe-critical legal domain, it is unclear how much legal knowledge they posses…

Articles Benchmarking Memorization Multi-Label Classification+1