Papers Language Model Evaluation
“Language Model Evaluation” 태그가 달린 논문 69편 · 필터 해제
Enterprise Large Language Model Evaluation Benchmark
Large Language Models (LLMs) ) have demonstrated promise in boosting productivity across AI-powered tools, yet existing benchmarks like Massive Multitask Language Understanding (MMLU) inadequately assess enterprise-speci…
Language Model EvaluationLanguage ModelingLanguage ModellingLarge Language Model+4Finance Language Model Evaluation (FLaME)
Language Models (LMs) have demonstrated impressive capabilities with core Natural Language Processing (NLP) tasks. The effectiveness of LMs for highly specialized knowledge-intensive tasks in finance remains difficult to…
BenchmarkingLanguage Model EvaluationLanguage ModelingLanguage Modelling+1BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models
Language model evaluation is a daunting task: prompts are brittle, corpus-level perplexities are vague, and the choice of benchmarks are endless. Finding examples that show meaningful, generalizable differences between t…
Language Model EvaluationFABLE: A Novel Data-Flow Analysis Benchmark on Procedural Text for Large Language Model Evaluation
Understanding how data moves, transforms, and persists, known as data flow, is fundamental to reasoning in procedural tasks. Despite their fluency in natural and programming languages, large language models (LLMs), altho…
DiagnosticLanguage Model EvaluationLanguage ModelingLanguage Modelling+1Role-Playing Evaluation for Large Language Models
Large Language Models (LLMs) demonstrate a notable capacity for adopting personas and engaging in role-playing. However, evaluating this ability presents significant challenges, as human assessments are resource-intensiv…
Language Model EvaluationR-Bench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation
Reasoning stands as a cornerstone of intelligence, enabling the synthesis of existing knowledge to solve complex problems. Despite remarkable progress, existing reasoning benchmarks often fail to rigorously evaluate the …
Language Model EvaluationLanguage ModelingLanguage ModellingMultimodal ReasoningConfidence in Large Language Model Evaluation: A Bayesian Approach to Limited-Sample Challenges
Large language models (LLMs) exhibit probabilistic output characteristics, yet conventional evaluation frameworks rely on deterministic scalar metrics. This study introduces a Bayesian approach for LLM capability assessm…
Bayesian InferenceLanguage Model EvaluationLanguage ModelingLanguage Modelling+1CoCo-Bench: A Comprehensive Code Benchmark For Multi-task Large Language Model Evaluation
Large language models (LLMs) play a crucial role in software engineering, excelling in tasks like code generation and maintenance. However, existing benchmarks are often narrow in scope, focusing on a specific task and l…
Code GenerationLanguage Model EvaluationLanguage ModelingLanguage Modelling+1UPME: An Unsupervised Peer Review Framework for Multimodal Large Language Model Evaluation
Multimodal Large Language Models (MLLMs) have emerged to tackle the challenges of Visual Question Answering (VQA), sparking a new research focus on conducting objective evaluations of these models. Existing evaluation me…
Language Model EvaluationLanguage ModelingLanguage ModellingLarge Language Model+4MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation
Traditional benchmarks struggle to evaluate increasingly sophisticated language models in multilingual and culturally diverse contexts. To address this gap, we introduce MMLU-ProX, a comprehensive multilingual benchmark …
Language Model EvaluationLanguage ModelingLanguage ModellingLarge Language Model+1Predicting Liquidity-Aware Bond Yields using Causal GANs and Deep Reinforcement Learning with LLM Evaluation
Financial bond yield forecasting is challenging due to data scarcity, nonlinear macroeconomic dependencies, and evolving market conditions. In this paper, we propose a novel framework that leverages Causal Generative Adv…
Decision MakingDeep Reinforcement LearningLanguage Model EvaluationLanguage Modeling+4M-ABSA: A Multilingual Dataset for Aspect-Based Sentiment Analysis
Aspect-based sentiment analysis (ABSA) is a crucial task in information extraction and sentiment analysis, aiming to identify aspects with associated sentiment elements in text. However, existing ABSA datasets are predom…
Aspect-Based Sentiment AnalysisAspect-Based Sentiment Analysis (ABSA)Language Model EvaluationLanguage Modeling+5Environmental large language model Evaluation (ELLE) dataset: A Benchmark for Evaluating Generative AI applications in Eco-environment Domain
Generative AI holds significant potential for ecological and environmental applications such as monitoring, data analysis, education, and policy support. However, its effectiveness is limited by the lack of a unified eva…
Language Model EvaluationLanguage ModelingLanguage ModellingLarge Language ModelAutomated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation
The rapid development of vision language models (VLMs) demands rigorous and reliable evaluation. However, current visual question answering (VQA) benchmarks often depend on open-ended questions, making accurate evaluatio…
Language Model EvaluationLanguage ModelingLanguage ModellingMultiple-choice+3Setting Standards in Turkish NLP: TR-MMLU for Large Language Model Evaluation
Language models have made remarkable advancements in understanding and generating human language, achieving notable success across a wide array of applications. However, evaluating these models remains a significant chal…
Language Model EvaluationLanguage ModelingLanguage ModellingLarge Language Model+2LMUnit: Fine-grained Evaluation with Natural Language Unit Tests
As language models become integral to critical workflows, assessing their behavior remains a fundamental challenge -- human evaluation is costly and noisy, while automated metrics provide only coarse, difficult-to-interp…
Language Model EvaluationLanguage ModelingLanguage ModellingTemplate Matters: Understanding the Role of Instruction Templates in Multimodal Language Model Evaluation and Training
Current multimodal language models (MLMs) evaluation and training approaches overlook the influence of instruction format, presenting an elephant-in-the-room problem. Previous research deals with this problem by manually…
Language Model EvaluationLanguage ModelingLanguage ModellingC$^2$LEVA: Toward Comprehensive and Contamination-Free Language Model Evaluation
Recent advances in large language models (LLMs) have shown significant promise, yet their evaluation raises concerns, particularly regarding data contamination due to the lack of access to proprietary training data. To a…
Language Model EvaluationLanguage ModelingLanguage ModellingDART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA
Recent advances in self-supervised models for natural language, vision, and protein sequences have inspired the development of large genomic DNA language models (DNALMs). These models aim to learn generalizable represent…
counterfactualLanguage Model EvaluationLanguage ModelingLanguage ModellingBenchmarking Harmonized Tariff Schedule Classification Models
The Harmonized Tariff System (HTS) classification industry, essential to e-commerce and international trade, currently lacks standardized benchmarks for evaluating the effectiveness of classification solutions. This stud…
BenchmarkingClassificationLanguage Model EvaluationLanguage Modeling+1