Papers HumanEval
“HumanEval” 태그가 달린 논문 264편 · 필터 해제
Turning the Tide: Repository-based Code Reflection
Code large language models (LLMs) enhance programming by understanding and generating code across languages, offering intelligent feedback, bug detection, and code updates through reflection, improving development effici…
Code GenerationDiversityHumanEvalRethinking Verification for LLM Code Generation: From Generation to Testing
Large language models (LLMs) have recently achieved notable success in code-generation benchmarks such as HumanEval and LiveCodeBench. However, a detailed examination reveals that these evaluation suites often comprise o…
Code GenerationHumanEvalany4: Learned 4-bit Numeric Representation for LLMs
We present any4, a learned 4-bit weight quantization solution for large language models (LLMs) providing arbitrary numeric representations without requiring pre-processing of weights or activations. any4 yields higher ac…
GPUGSM8KHumanEvalmbpp+2SACL: Understanding and Combating Textual Bias in Code Retrieval with Semantic-Augmented Reranking and Localization
Retrieval-Augmented Code Generation (RACG) is a critical technique for enhancing code generation by retrieving relevant information. In this work, we conduct an in-depth analysis of code retrieval by systematically maski…
Code GenerationHumanEvalmbppReranking+1Plan for Speed -- Dilated Scheduling for Masked Diffusion Language Models
Masked diffusion language models (MDLM) have shown strong promise for non-autoregressive text generation, yet existing samplers act as implicit planners, selecting tokens to unmask via denoiser confidence or entropy scor…
Code CompletionGSM8KHumanEvalMath+3AgentGroupChat-V2: Divide-and-Conquer Is What LLM-Based Multi-Agent System Need
Large language model based multi-agent systems have demonstrated significant potential in social simulation and complex task resolution domains. However, current frameworks face critical challenges in system architecture…
GSM8KHumanEvalLarge Language ModelMath+1Guaranteed Guess: A Language Modeling Approach for CISC-to-RISC Transpilation with Testing Guarantees
The hardware ecosystem is rapidly evolving, with increasing interest in translating low-level programs across different instruction set architectures (ISAs) in a quick, flexible, and correct way to enhance the portabilit…
Code TranslationHumanEvalLanguage ModelingLanguage Modelling+2LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing
Recent efforts to combine low-rank adaptation (LoRA) with mixture-of-experts (MoE) for adapting large language models (LLMs) to multiple tasks still exhibit prevailing limitations: they either swap entire attention/feed-…
ARCCoLAGSM8KHumanEval+4Guideline Forest: Experience-Induced Multi-Guideline Reasoning with Stepwise Aggregation
Human reasoning is flexible, adaptive, and grounded in prior experience-qualities that large language models (LLMs) still struggle to emulate. Existing methods either explore diverse reasoning paths at inference time or …
GSM8KHumanEvalMathmbppSwiftEval: Developing a Language-Specific Benchmark for LLM-generated Code Evaluation
In recent years, large language models (LLMs) have showcased significant advancements in code generation. However, most evaluation benchmarks are primarily oriented towards Python, making it difficult to evaluate other p…
Code GenerationHumanEvalSelf-Correcting Code Generation Using Small Language Models
Self-correction has demonstrated potential in code generation by allowing language models to revise and improve their outputs through successive refinement. Recent studies have explored prompting-based strategies that in…
Code GenerationHumanEvalmbppEnhancing LLM-Based Code Generation with Complexity Metrics: A Feedback-Driven Approach
Automatic code generation has gained significant momentum with the advent of Large Language Models (LLMs) such as GPT-4. Although many studies focus on improving the effectiveness of LLMs for code generation, very limite…
Code GenerationHumanEvalmbppActor-Critic based Online Data Mixing For Language Model Pre-Training
The coverage and composition of pretraining data significantly impacts the generalization ability of Large Language Models (LLMs). To reduce the carbon footprint and financial costs of training, some data mixing methods,…
HumanEvalLanguage ModelingLanguage ModellingMMLUAn LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks
Large Language Models (LLMs) and other automated techniques have been increasingly used to support software developers by generating software artifacts such as code snippets, patches, and comments. However, accurately as…
Code GenerationCode SummarizationHumanEvalProgram RepairEvaluating Large Language Models for Code Review
Context: Code reviews are crucial for software quality. Recent AI advances have allowed large language models (LLMs) to review and fix code; now, there are tools that perform these reviews. However, their reliability and…
HumanEvalLLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models
While Masked Diffusion Models (MDMs), such as LLaDA, present a promising paradigm for language modeling, there has been relatively little effort in aligning these models with human preferences via reinforcement learning.…
GSM8KHumanEvalLanguage ModelingLanguage Modelling+1From Output to Evaluation: Does Raw Instruction-Tuned Code LLMs Output Suffice for Fill-in-the-Middle Code Generation?
Post-processing is crucial for the automatic evaluation of LLMs in fill-in-the-middle (FIM) code generation due to the frequent presence of extraneous code in raw outputs. This extraneous generation suggests a lack of aw…
Code GenerationHumanEvalInvisible Entropy: Towards Safe and Efficient Low-Entropy LLM Watermarking
Logit-based LLM watermarking traces and verifies AI-generated content by maintaining green and red token lists and increasing the likelihood of green tokens during generation. However, it fails in low-entropy scenarios, …
HumanEvalmbppPrior Prompt Engineering for Reinforcement Fine-Tuning
This paper investigates prior prompt engineering (pPE) in the context of reinforcement fine-tuning (RFT), where language models (LMs) are incentivized to exhibit behaviors that maximize performance through reward signals…
HumanEvalPrompt EngineeringWarm Up Before You Train: Unlocking General Reasoning in Resource-Constrained Settings
Designing effective reasoning-capable LLMs typically requires training using Reinforcement Learning with Verifiable Rewards (RLVR) or distillation with carefully curated Long Chain of Thoughts (CoT), both of which depend…
HumanEvalMathMMLU