paper-with-me

Papers

OriGen:Enhancing RTL Code Generation with Code-to-Code Augmentation and Self-Reflection

2024-07-23 · Fan Cui, Chenyang Yin, Kexing Zhou, Youwei Xiao, Guangyu Sun, Qiang Xu, Qipeng Guo, Demin Song, Dahua Lin, Xingcheng Zhang, Yun, Liang

Recent studies have demonstrated the significant potential of Large Language Models (LLMs) in generating Register Transfer Level (RTL) code, with notable advancements showcased by commercial models such as GPT-4 and Claude3-Opus. However, these proprietary LLMs often raise concerns regarding privacy and security. While open-source LLMs offer solutions to these concerns, they typically underperform commercial models in RTL code generation tasks, primarily due to the scarcity of high-quality open-source RTL datasets. To address this challenge, we introduce OriGen , a fully open-source framework that incorporates self-reflection capabilities and a novel dataset augmentation methodology for generating high-quality, large-scale RTL code. Our approach employs a code-tocode augmentation technique to enhance the quality of open-source RTL code datasets. Furthermore, OriGen can rectify syntactic errors through a self-reflection process that leverages compiler feedback. Experimental results demonstrate that OriGen significantly outperforms other open-source alternatives in RTL code generation. It surpasses the previous best-performing open-source LLM by 12.8% and even exceeds GPT-4 Turbo in the pass@1 metric on the VerilogEval-Human benchmark. Moreover, OriGen exhibits superior capabilities in self-reflection and error correction, outperforming GPT-4 by 19.9% on a benchmark designed to evaluate self-reflection capabilities.

📄 PDF Abstract BibTeX arXiv:2407.16237

Code (1)

pku-liang/origen 공식 구현 pytorch

Tasks

Code GenerationKnowledge Distillation

Methods 이 논문이 사용한 방법론

Attention 설명 없음
Adam 설명 없음
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Position-Wise Feed-Forward Layer 설명 없음

Similar Papers 제목 키워드 기반

ORIGEN: Zero-Shot 3D Orientation Grounding in Text-to-Image Generation

2025-03-28 · Yunhong Min, Daehyeon Choi, Kyeongmin Yeo, Jihyun Lee 외

We introduce ORIGEN, the first zero-shot method for 3D orientation grounding in text-to-image generation across multiple objects and diverse categories. While previous work on spatial grounding in image generation has ma…

Image GenerationText to Image GenerationText-to-Image Generation

Breaking Memorization Barriers in LLM Code Fine-Tuning via Information Bottleneck for Improved Generalization

2025-10-15 · Changsheng Wang, Xin Chen, Sijia Liu, Ke Ding arxiv

Adapting pretrained large language models (LLMs) to code domains via supervised fine-tuning (FT) has been commonly used for code generation. However, we identify a previously underappreciated failure mode, the memorizati…

Code Generation

Florigens and antiflorigens: a molecular genetic understanding

2015-09-21

Florigens, the leaf-derived signals that initiate flowering, have been described as mysterious, elusive and the Holy Grail of plant biology.

GeoCode-GPT: A Large Language Model for Geospatial Code Generation Tasks

2024-10-22 · Shuyang Hou, Zhangxiao Shen, Anqi Zhao, Jianyuan Liang 외

The increasing demand for spatiotemporal data and modeling tasks in geosciences has made geospatial code generation technology a critical factor in enhancing productivity. Although large language models (LLMs) have demon…

Code GenerationCode SummarizationHallucinationLanguage Modeling+4

Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey

2024-12-29 · Junqiao Wang, Zeng Zhang, Yangfan He, Zihao Zhang 외

Reinforcement learning (RL) has emerged as a powerful paradigm for enhancing large language models (LLMs) in code generation and optimization. This survey systematically reviews RL-driven techniques across the code devel…

Code GenerationCompiler Optimizationreinforcement-learningReinforcement Learning+2