The Heap: A Contamination-Free Multilingual Code Dataset for Evaluating Large Language Models
The recent rise in the popularity of large language models has spurred the development of extensive code datasets needed to train them. This has left limited code available for collection and use in the downstream investigation of specific behaviors, or evaluation of large language models without suffering from data contamination. To address this problem, we release The Heap, a large multilingual dataset covering 57 programming languages that has been deduplicated with respect to other open datasets of code, enabling researchers to conduct fair evaluations of large language models without significant data cleaning overhead.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Contamination Report for Multilingual Benchmarks
Benchmark contamination refers to the presence of test datasets in Large Language Model (LLM) pre-training or post-training data. Contamination can lead to inflated scores on benchmarks, compromising evaluation results a…
Language ModelingLanguage ModellingLarge Language ModelObscuring Data Contamination Through Translation: Evidence from Arabic Corpora
Data contamination undermines the validity of Large Language Model evaluation by enabling models to rely on memorized benchmark content rather than true generalization. While prior work has proposed contamination detecti…
A Novel Heap-based Pilot Assignment for Full Duplex Cell-Free Massive MIMO with Zero-Forcing
This paper investigates the combined benefits of full-duplex (FD) and cell-free massive multiple-input multipleoutput (CF-mMIMO), where a large number of distributed access points (APs) having FD capability simultaneousl…
Robust DesignData Contamination Can Cross Language Barriers
The opacity in developing large language models (LLMs) is raising growing concerns about the potential contamination of public benchmarks in the pre-training data. Existing contamination detection methods are typically b…
MemorizationLNE-Blocking: An Efficient Framework for Contamination Mitigation Evaluation on Large Language Models
The problem of data contamination is now almost inevitable during the development of large language models (LLMs), with the training data commonly integrating those evaluation benchmarks even unintentionally. This proble…