Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies
Thematic analysis and other variants of inductive coding are widely used qualitative analytic methods within empirical legal studies (ELS). We propose a novel framework facilitating effective collaboration of a legal expert with a large language model (LLM) for generating initial codes (phase 2 of thematic analysis), searching for themes (phase 3), and classifying the data in terms of the themes (to kick-start phase 4). We employed the framework for an analysis of a dataset (n=785) of facts descriptions from criminal court opinions regarding thefts. The goal of the analysis was to discover classes of typical thefts. Our results show that the LLM, namely OpenAI's GPT-4, generated reasonable initial codes, and it was capable of improving the quality of the codes based on expert feedback. They also suggest that the model performed well in zero-shot classification of facts descriptions in terms of the themes. Finally, the themes autonomously discovered by the LLM appear to map fairly well to the themes arrived at by legal experts. These findings can be leveraged by legal researchers to guide their decisions in integrating LLMs into their thematic analyses, as well as other inductive coding projects.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModellingLarge Language Modelzero-shot-classificationZero-Shot LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
How Numerical Precision Affects Mathematical Reasoning Capabilities of LLMs
Despite the remarkable success of Transformer-based Large Language Models (LLMs) across various domains, understanding and enhancing their mathematical capabilities remains a significant challenge. In this paper, we cond…
Mathematical ReasoningThematic Analysis with Large Language Models: does it work with languages other than English? A targeted test in Italian
This paper proposes a test to perform Thematic Analysis (TA) with Large Language Model (LLM) on data which is in a different language than English. While there has been initial promising work on using pre-trained LLMs fo…
Language ModelingLanguage ModellingLarge Language ModelAutomating Thematic Analysis: How LLMs Analyse Controversial Topics
Large Language Models (LLMs) are promising analytical tools. They can augment human epistemic, cognitive and reasoning abilities, and support 'sensemaking', making sense of a complex environment or subject by analysing l…
Position: Thematic Analysis of Unstructured Clinical Transcripts with Large Language Models
This position paper examines how large language models (LLMs) can support thematic analysis of unstructured clinical transcripts, a widely used but resource-intensive method for uncovering patterns in patient and provide…
Designing an Auditable LLM-Supported Workflow for Qualitative Thematic Analysis
Large Language Models (LLMs) offer new possibilities for scaling qualitative analysis, but existing applications often provide limited methodological transparency regarding how qualitative methods are translated into com…
Domain Adaptation