Bench4HLS: End-to-End Evaluation of LLMs in High-Level Synthesis Code Generation

2026-01-16 · M Zafir Sadik Khan, Kimia Azar, Hadi Kamali arxiv

In last two years, large language models (LLMs) have shown strong capabilities in code generation, including hardware design at register-transfer level (RTL). While their use in high-level synthesis (HLS) remains comparatively less mature, the ratio of HLS- to RTL-focused studies has shifted from 1:10 to 2:10 in the past six months, indicating growing interest in leveraging LLMs for high-level design entry while relying on downstream synthesis for optimization. This growing trend highlights the need for a comprehensive benchmarking and evaluation framework dedicated to LLM-based HLS. To address this, We present Bench4HLS for evaluating LLM-generated HLS designs. Bench4HLS comprises 170 manually drafted and validated case studies, spanning small kernels to complex accelerators, curated from widely used public repositories. The framework supports fully automated assessment of compilation success, functional correctness via simulation, and synthesis feasibility/optimization. Crucially, Bench4HLS integrates a pluggable API for power, performance, and area (PPA) analysis across various HLS toolchains and architectures, demonstrated here with Xilinx Vitis HLS and validated on Catapult HLS. By providing a structured, extensible, and plug-and-play testbed, Bench4HLS establishes a foundational methodology for benchmarking LLMs in HLS workflows.

📄 PDF Abstract BibTeX arXiv:2601.19941

Code (0)

등록된 구현이 없습니다.

Tasks

Code Generation

LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation

2023-05-18 · NeurIPS 2023 11 · Yujie Lu, Xianjun Yang, Xiujun Li, Xin Eric Wang 외

Existing automatic evaluation on text-to-image synthesis can only provide an image-text matching score, without considering the object-level compositionality, which results in poor correlation with human judgments. In th…

Attribute Image Generation Image-text matching Text Matching

HLS-Eval: A Benchmark and Framework for Evaluating LLMs on High-Level Synthesis Design Tasks

2025-04-16 · Stefan Abi-Karam, Cong Hao

The rapid scaling of large language model (LLM) training and inference has driven their adoption in semiconductor design across academia and industry. While most prior work evaluates LLMs on hardware description language…

High-Level Synthesis Large Language Model

URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment

2026-07-06 · Bogdan Zagribelnyy, Ivan Ilin, Nikita Bondarev, Anton Morgunov 외 arxiv

Synthesis planning aiming to find pathways of reactions for a target molecule is one of the most important and challenging tasks in drug discovery. Recent progress has produced both specialized deep-learning retrosynthes…

Drug Discovery

VGenST-Bench: A Benchmark for Spatio-Temporal Reasoning via Active Video Synthesis

2026-05-21 · Jinho Park, Youbin Kim, Hogun Park, Eunbyung Park arxiv

Spatio-temporal reasoning is a core capability for Multimodal Large Language Models (MLLMs) operating in the real world. As such, evaluating it precisely has become an essential challenge. However, existing spatio-tempor…

HardSecBench: Benchmarking the Security Awareness of LLMs for Hardware Code Generation

2026-01-20 · Qirui Chen, Jingxian Shuai, Shuangwu Chen, Shenghao Ye 외 arxiv

Large language models (LLMs) are increasingly used for hardware and firmware code generation, but existing studies primarily evaluate functional correctness while largely overlooking security. However, LLM-generated code…