GANzzle: Reframing jigsaw puzzle solving as a retrieval task using a generative mental image
Puzzle solving is a combinatorial challenge due to the difficulty of matching adjacent pieces. Instead, we infer a mental image from all pieces, which a given piece can then be matched against avoiding the combinatorial explosion. Exploiting advancements in Generative Adversarial methods, we learn how to reconstruct the image given a set of unordered pieces, allowing the model to learn a joint embedding space to match an encoding of each piece to the cropped layer of the generator. Therefore we frame the problem as a R@1 retrieval task, and then solve the linear assignment using differentiable Hungarian attention, making the process end-to-end. In doing so our model is puzzle size agnostic, in contrast to prior deep learning methods which are single size. We evaluate on two new large-scale datasets, where our model is on par with deep learning methods, while generalizing to multiple puzzle sizes.
Code (1)
Tasks
Deep LearningRetrievalSimilar Papers 제목 키워드 기반
Solving Mixed-Modal Jigsaw Puzzle for Fine-Grained Sketch-Based Image Retrieval
ImageNet pre-training has long been considered crucial by the fine-grained sketch-based image retrieval (FG-SBIR) community due to the lack of large sketch-photo paired datasets for FG-SBIR training. In this paper, we pr…
Image RetrievalRetrievalSketch-Based Image RetrievalThe Missing GAP: From Solving Square Jigsaw Puzzles to Handling Real World Archaeological Fragments
Jigsaw puzzle solving has been an increasingly popular task in the computer vision research community. Recent works have utilized cutting-edge architectures and computational approaches to reassemble groups of pieces int…
Jigsaw-ViT: Learning Jigsaw Puzzles in Vision Transformer
The success of Vision Transformer (ViT) in various computer vision tasks has promoted the ever-increasing prevalence of this convolution-free network. The fact that ViT works on image patches makes it potentially relevan…
ClassificationDomain Generalizationimage-classificationImage Classification+2JigsawGAN: Auxiliary Learning for Solving Jigsaw Puzzles with Generative Adversarial Networks
The paper proposes a solution based on Generative Adversarial Network (GAN) for solving jigsaw puzzles. The problem assumes that an image is divided into equal square pieces, and asks to recover the image according to in…
Auxiliary LearningGeneral ClassificationGenerative Adversarial NetworkSelf-Supervised LearningSolving Multiple Square Jigsaw Puzzles With Missing Pieces
Jigsaw-puzzle solving is necessary in many applications, including biology, archaeology, and every-day life. In this paper we consider the square jigsaw puzzle problem, where the goal is to reconstruct the image from a s…