paper-with-me

Papers

Learning to Solve Constraint Satisfaction Problems with Recurrent Transformer

2023-07-10 · Zhun Yang, Adam Ishay, Joohyung Lee

Constraint satisfaction problems (CSPs) are about finding values of variables that satisfy the given constraints. We show that Transformer extended with recurrence is a viable approach to learning to solve CSPs in an end-to-end manner, having clear advantages over state-of-the-art methods such as Graph Neural Networks, SATNet, and some neuro-symbolic models. With the ability of Transformer to handle visual input, the proposed Recurrent Transformer can straightforwardly be applied to visual constraint reasoning problems while successfully addressing the symbol grounding problem. We also show how to leverage deductive knowledge of discrete constraints in the Transformer's inductive learning to achieve sample-efficient learning and semi-supervised learning for CSPs.

📄 PDF Abstract BibTeX arXiv:2307.04895

Code (1)

azreasoners/recurrent_transformer 공식 구현 pytorch

Tasks

Inductive Learning

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Adam 설명 없음
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Similar Papers 제목 키워드 기반

Recurrent networks of coupled Winner-Take-All oscillators for solving constraint satisfaction problems

2013-12-01 · NeurIPS 2013 12 · Hesham Mostafa, Lorenz. K. Mueller, Giacomo Indiveri

We present a recurrent neuronal network, modeled as a continuous-time dynamical system, that can solve constraint satisfaction problems. Discrete variables are represented by coupled Winner-Take-All (WTA) networks, and t…

All

Adapting Stochastic Search For Real-time Dynamic Weighted Constraint Satisfaction

2015-07-22 · Gregory Hasseler

This work presents two new algorithms for performing constraint satisfaction. The first algorithm presented, DMaxWalkSat, is a constraint solver specialized for solving dynamic, weighted constraint satisfaction problems.…

Guided Discrete Diffusion for Constraint Satisfaction Problems

2025-12-16 · Justin Jung arxiv

We propose discrete diffusion guidance for constraint satisfaction problems (CSPs) and demonstrate its ability to solve Sudoku puzzles without supervision.

Solving constraint-satisfaction problems with distributed neocortical-like neuronal networks

2018-01-14

Finding actions that satisfy the constraints imposed by both external inputs and internal representations is central to decision making. We demonstrate that some important classes of constraint satisfaction problems (CSP…

Decision MakingMathematical Proofs

Transformers satisfy

2021-01-01 · Feng Shi, Chen Li, Shijie Bian, Yiqiao Jin 외

The Propositional Satisfiability Problem (SAT), and more generally, the Constraint Satisfaction Problem (CSP), are mathematical questions defined as finding an assignment to a set of objects that satisfies a series of co…