CodeXGLUE

홈페이지 · 논문 205편

CodeXGLUE is a benchmark dataset and open challenge for code intelligence. It includes a collection of code intelligence tasks and a platform for model evaluation and comparison. CodeXGLUE stands for General Language Understanding Evaluation benchmark for CODE. It includes 14 datasets for 10 diversified code intelligence tasks covering the following scenarios: - code-code (clone detection, defect detection, cloze test, code completion, code repair, and code-to-code translation) - text-code (natural language code search, text-to-code generation) - code-text (code summarization) - text-text (documentation translation) A brief summary of CodeXGLUE is provided in the figure, including tasks, datasets, language, sizes in various states, baseline systems, providers, and short definitions of each task. Datasets highlighted in BLUE are newly introduced. Image source: https://github.com/microsoft/CodeXGLUE

Texts

벤치마크

Text-to-Code Generation on CodeXGLUE - CONCODE 결과 4개

Code Completion on CodeXGLUE - Github Java Corpus 결과 3개

Code Completion on CodeXGLUE - PY150 결과 3개

Code Search on CodeXGLUE - AdvTest 결과 3개

Code Generation on CodeXGLUE - CodeSearchNet 결과 2개

Code Repair on CodeXGLUE - Bugs2Fix 결과 2개

Code Translation on CodeXGLUE - CodeTrans 결과 2개

Cloze Test on CodeXGLUE - CT-all 결과 1개

Cloze Test on CodeXGLUE - CT-maxmin 결과 1개

Code Search on CodeXGLUE - WebQueryTest 결과 1개