paper-with-me

Papers

Circuit Transformer: A Transformer That Preserves Logical Equivalence

2024-03-14 · Xihan Li, Xing Li, Lei Chen, Xing Zhang, Mingxuan Yuan, Jun Wang

Implementing Boolean functions with circuits consisting of logic gates is fundamental in digital computer design. However, the implemented circuit must be exactly equivalent, which hinders generative neural approaches on this task due to their occasionally wrong predictions. In this study, we introduce a generative neural model, the "Circuit Transformer", which eliminates such wrong predictions and produces logic circuits strictly equivalent to given Boolean functions. The main idea is a carefully designed decoding mechanism that builds a circuit step-by-step by generating tokens, which has beneficial "cutoff properties" that block a candidate token once it invalidate equivalence. In such a way, the proposed model works similar to typical LLMs while logical equivalence is strictly preserved. A Markov decision process formulation is also proposed for optimizing certain objectives of circuits. Experimentally, we trained an 88-million-parameter Circuit Transformer to generate equivalent yet more compact forms of input circuits, outperforming existing neural approaches on both synthetic and real world benchmarks, without any violation of equivalence constraints.

📄 PDF Abstract BibTeX arXiv:2403.13838

Code (1)

snowkylin/circuit-transformer 공식 구현 tf

Tasks

Hallucination

Methods 이 논문이 사용한 방법론

Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Multi-Head Attention 설명 없음
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Similar Papers 제목 키워드 기반

When Close Enough Is Not Enough: Autoregressive Drift in Quantum Circuit Synthesis

2026-07-14 · Mehdi Saeedi, Eddie Richter, Paul Hartke arxiv

Quantum circuit optimization for fault-tolerant computing requires exact functional equivalence while minimizing expensive non-Clifford resources such as T gates. We study this problem using a compact 44.8M-parameter enc…

Logic Synthesis with Generative Deep Neural Networks

2024-06-07 · Xihan Li, Xing Li, Lei Chen, Xing Zhang 외

While deep learning has achieved significant success in various domains, its application to logic circuit design has been limited due to complex constraints and strict feasibility requirement. However, a recent generativ…

Revisiting Padded Transformer Expressivity: Which Architectural Choices Matter and Which Don't

2026-05-28 · Anej Svete, William Merrill, Ryan Cotterell, Ashish Sabharwal arxiv

Recent work describes what transformers can and cannot compute through connections to boolean circuits, but existing results lack exact characterizations and are sensitive to modeling choices. Padded transformers -- to w…

GTAC: A Generative Transformer for Approximate Circuits

2025-12-08 · Jingxin Wang, Shitong Guo, Wenhui Liang, Ruicheng Dai 외 arxiv

Targeting error-tolerant applications, approximate computing relaxes rigid functional equivalence to significantly improve power, performance, and area. Traditional approximate logic synthesis (ALS) relies on incremental…

Training Trajectories Determine Circuit Removability in Annealable Soft-Prior Transformers

2026-09-09 · Zonglin Yang, Ziming Zhao, Wei Tang, Xunyu Jiang 외 arxiv

Soft positional priors can help small Transformers learn retrieval circuits, but it is unclear whether the resulting circuits remain functional once the prior is removed. We test this with an annealable soft-prior Transf…