LexFlip: A Dissociation Diagnostic for Legal Meaning Preservation Metrics
Does a simplified legal clause still say what the original said? The checks in current use cannot establish that it does: requiring an identical pair to score highest and an unrelated pair lowest moves lexical overlap and legal force together, so any monotone function of token overlap satisfies both. Our remedy is a dissociation, an item holding surface form fixed while legal force moves. We release LexFlip, 373 minimal perturbations of Quebec statutory French that reverse legal force while preserving 0.93 of the tokens, with a harness scoring metrics, regressors and prompted judges alike. The seven embedding and BERTScore metrics we test spend only 0.022 to 0.039 of their identical-to-unrelated range on such an edit, against 0.670 for bidirectional NLI, the one family the identical-pair check would disqualify. On FrJudge, against a measured human ceiling of r=0.597, a bare length feature outscores every semantic metric and has the lowest margin we measure.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
JUDGEBERT: Assessing Legal Meaning Preservation Between Sentences
Simplifying text while preserving its meaning is a complex yet essential task, especially in sensitive domain applications like legal texts. When applied to a specialized field, like the legal domain, preservation differ…
Text SimplificationTowards a general model for psychopathology
The DSM-1 was published in 1952, contains 128 diagnostic categories, described in 132 pages. The DSM-5 appeared in 2013, contains 541 diagnostic categories, described in 947 pages. The field of psychology is characterise…
DiagnosticmodelLexRubric: A Rubric-Guided Diagnostic Benchmark for Open-Ended Legal Tasks
As large language models (LLMs) are increasingly applied to real-world legal tasks, evaluating the reliability of their open-ended legal responses has become essential. These tasks require context-sensitive answers and a…
Legal ReasoningFunctional dissociations versus post-hoc selection: Moving beyond the Stockart et al. (2025) compromise
Stockart et al. (2025) recommend guidelines for best practices in the field of unconscious cognition. However, they condone the repeatedly criticized technique of excluding trials with high visibility ratings or of parti…
SensitivityContrastive-Difference CKA Reveals Concept-Specific Structural Alignment Across Language Model Architectures
Do different LLM architectures encode high-level concepts in structurally compatible ways? We systematically characterize a geometric-functional universality dissociation: across multiple concept domains and architectura…