홈 › Papers

Qiskit HumanEval: An Evaluation Benchmark For Quantum Code Generative Models

2024-06-20 · Sanjay Vishwakarma, Francis Harkins, Siddharth Golecha, Vishal Sharathchandra Bajpe, Nicolas Dupuis, Luca Buratti, David Kremer, Ismael Faro, Ruchir Puri, Juan Cruz-Benito

Quantum programs are typically developed using quantum Software Development Kits (SDKs). The rapid advancement of quantum computing necessitates new tools to streamline this development process, and one such tool could be Generative Artificial intelligence (GenAI). In this study, we introduce and use the Qiskit HumanEval dataset, a hand-curated collection of tasks designed to benchmark the ability of Large Language Models (LLMs) to produce quantum code using Qiskit - a quantum SDK. This dataset consists of more than 100 quantum computing tasks, each accompanied by a prompt, a canonical solution, a comprehensive test case, and a difficulty scale to evaluate the correctness of the generated solutions. We systematically assess the performance of a set of LLMs against the Qiskit HumanEval dataset's tasks and focus on the models ability in producing executable quantum code. Our findings not only demonstrate the feasibility of using LLMs for generating quantum code but also establish a new benchmark for ongoing advancements in the field and encourage further exploration and development of GenAI-driven tools for quantum code generation.

📄 PDF Abstract BibTeX arXiv:2406.14712

Code (0)

등록된 구현이 없습니다.

Tasks

Code GenerationHumanEval

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Focus 설명 없음

Quantum Verifiable Rewards for Post-Training Qiskit Code Assistant

2025-08-28 · Nicolas Dupuis, Adarsh Tiwari, Youssef Mroueh, David Kremer 외 arxiv

Qiskit is an open-source quantum computing framework that allows users to design, simulate, and run quantum circuits on real quantum hardware. We explore post-training techniques for LLMs to assist in writing Qiskit code…

QSpark: Towards Reliable Qiskit Code Generation

2025-07-16 · Kiana Kheiri, Aamna Aamir, Andriy Miranskyy, Chen Ding arxiv

Quantum circuits must be error-resilient, yet LLMs like Granite-20B-Code and StarCoder often output flawed Qiskit code. We fine-tuned the Qwen2.5-Coder-32B model with two RL methods, Group Relative Policy Optimization (G…

Code Generation

Qiskit Code Assistant: Training LLMs for generating Quantum Computing Code

2024-05-29 · Nicolas Dupuis, Luca Buratti, Sanjay Vishwakarma, Aitana Viudes Forrat 외

Code Large Language Models (Code LLMs) have emerged as powerful tools, revolutionizing the software development landscape by automating the coding process and reducing time and effort required to build applications. This…

HumanEval

Revisiting Quantum Code Generation: Where Should Domain Knowledge Live?

2026-03-23 · Oscar Novo, Oscar Bastidas-Jossa, Alberto Calvo, Antonio Peris 외 arxiv

Recent advances in large language models (LLMs) have enabled the automation of an increasing number of programming tasks, including code generation for scientific and engineering domains. In rapidly evolving software eco…

Code Generation

Generative AI for Quantum Circuits and Quantum Code: A Technical Review and Taxonomy

2026-03-17 · Juhani Merilehto arxiv

We review thirteen generative systems and five supporting datasets for quantum circuit and quantum code generation, identified through a structured scoping review of Hugging Face, arXiv, and provenance tracing (January-F…

Graph Generation Code Generation