T-FIX: Text-Based Explanations with Features Interpretable to eXperts
As LLMs are deployed in knowledge-intensive settings (e.g., surgery, astronomy, therapy), users are often domain experts who expect not just answers, but explanations that mirror professional reasoning. Yet evaluating whether an LLM "thinks like an expert" remains difficult: existing approaches rely on per-example expert annotation, making them costly, hard to scale, and tied to a single notion of correct reasoning within each domain. To address this gap, we introduce T-FIX, a unified evaluation framework that operationalizes expert alignment as a desired attribute of LLM-generated explanations. T-FIX spans seven scientific tasks across three domains, with each task evaluated against expert-defined criteria that capture domain-grounded reasoning rather than generic explanation quality. Our framework enables automatic, personalizable evaluation of expert alignment that generalizes to unseen explanations without ongoing expert involvement. Code is available at https://github.com/BrachioLab/FIX-2/.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Intrinsic User-Centric Interpretability through Global Mixture of Experts
In human-centric settings like education or healthcare, model accuracy and model explainability are key factors for user adoption. Towards these two goals, intrinsically interpretable deep learning models have gained pop…
Mixture-of-ExpertsNews ClassificationInterpretable Mixture of Experts
The need for reliable model explanations is prominent for many machine learning applications, particularly for tabular and time-series data as their use cases often involve high-stakes decision making. Towards this goal,…
Decision MakingMixture-of-ExpertsTime SeriesThe Need for Interpretable Features: Motivation and Taxonomy
Through extensive experience developing and explaining machine learning (ML) applications for real-world domains, we have learned that ML models are only as interpretable as their features. Even simple, highly interpreta…
Decision MakingMind the XAI Gap: A Human-Centered LLM Framework for Democratizing Explainable AI
Artificial Intelligence (AI) is rapidly embedded in critical decision-making systems, however their foundational ``black-box'' models require eXplainable AI (XAI) solutions to enhance transparency, which are mostly orien…
BenchmarkingIn-Context LearningExplainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery
With the growing capabilities of intelligent systems, the integration of robots in our everyday life is increasing. However, when interacting in such complex human environments, the occasional failure of robotic systems …
Decision MakingDecoder