HumanEval

홈페이지 · 논문 1,201편

This is an evaluation harness for the HumanEval problem solving dataset described in the paper "Evaluating Large Language Models Trained on Code". It used to measure functional correctness for synthesizing programs from docstrings. It consists of 164 original programming problems, assessing language comprehension, algorithms, and simple mathematics, with some comparable to simple software interview questions. Source: [Evaluating Large Language Models Trained on Code](/paper/evaluating-large-language-models-trained-on) Image Source: [Evaluating Large Language Models Trained on Code](/paper/evaluating-large-language-models-trained-on)

벤치마크