Preliminary WMT24 Ranking of General MT Systems and LLMs
This is the preliminary ranking of WMT24 General MT systems based on automatic metrics. The official ranking will be a human evaluation, which is superior to the automatic ranking and supersedes it. The purpose of this report is not to interpret any findings but only provide preliminary results to the participants of the General MT task that may be useful during the writing of the system submission.
Code (1)
Similar Papers 제목 키워드 기반
Preliminary Ranking of WMT25 General Machine Translation Systems
We present the preliminary rankings of machine translation (MT) systems submitted to the WMT25 General Machine Translation Shared Task, as determined by automatic evaluation metrics. Because these rankings are derived fr…
Machine TranslationThe Ranking Blind Spot: Decision Hijacking in LLM-based Text Ranking
Large Language Models (LLMs) have demonstrated strong performance in information retrieval tasks like passage ranking. Our research examines how instruction-following capabilities in LLMs interact with multi-document com…
Information RetrievalPassage RankingPrompting for Performance: Exploring LLMs for Configuring Software
Software systems usually provide numerous configuration options that can affect performance metrics such as execution time, memory usage, binary size, or bitrate. On the one hand, making informed decisions is challenging…
Large Language Models are Zero-Shot Rankers for Recommender Systems
Recently, large language models (LLMs) (e.g., GPT-4) have demonstrated impressive general-purpose task-solving abilities, including the potential to approach recommendation tasks. Along this line of research, this work a…
Recommendation SystemsCRS Arena: Crowdsourced Benchmarking of Conversational Recommender Systems
We introduce CRS Arena, a research platform for scalable benchmarking of Conversational Recommender Systems (CRS) based on human feedback. The platform displays pairwise battles between anonymous conversational recommend…
BenchmarkingRecommendation Systems