The Collapse of Patches
Observing certain patches in an image reduces the uncertainty of others. Their realization lowers the distribution entropy of each remaining patch feature, analogous to collapsing a particle's wave function in quantum mechanics. This phenomenon can intuitively be called patch collapse. To identify which patches are most relied on during a target region's collapse, we learn an autoencoder that softly selects a subset of patches to reconstruct each target patch. Graphing these learned dependencies for each patch's PageRank score reveals the optimal patch order to realize an image. We show that respecting this order benefits various masked image modeling methods. First, autoregressive image generation can be boosted by retraining the state-of-the-art model MAR. Next, we introduce a new setup for image classification by exposing Vision Transformers only to high-rank patches in the collapse order. Seeing 22\% of such patches is sufficient to achieve high accuracy. With these experiments, we propose patch collapse as a novel image modeling perspective that promotes vision efficiency. Our project is available at https://github.com/wguo-ai/CoP .
Code (0)
등록된 구현이 없습니다.
Tasks
Image ClassificationImage GenerationSimilar Papers 제목 키워드 기반
Shape Reconstruction by Learning Differentiable Surface Representations
Generative models that produce point clouds have emerged as a powerful tool to represent 3D surfaces, and the best current ones rely on learning an ensemble of parametric representations. Unfortunately, they offer no con…
EXAONEPath 1.0 Patch-level Foundation Model for Pathology
Recent advancements in digital pathology have led to the development of numerous foundational models that utilize self-supervised learning on patches extracted from gigapixel whole slide images (WSIs). While this approac…
modelSelf-Supervised Learningwhole slide imagesEdge fires drive the shape and stability of tropical forests
In tropical regions, fires propagate readily in grasslands but typically consume only edges of forest patches. Thus forest patches grow due to tree propagation and shrink by fires in surrounding grasslands. The interplay…
RelationEnhancing Masked Time-Series Modeling via Dropping Patches
This paper explores how to enhance existing masked time-series modeling by randomly dropping sub-sequence level patches of time series. On this basis, a simple yet effective method named DropPatch is proposed, which has …
Cross-Domain Few-Shotcross-domain few-shot learningData AugmentationFew-Shot Learning+1Sparse Generative Adversarial Network
We propose a new approach to Generative Adversarial Networks (GANs) to achieve an improved performance with additional robustness to its so-called and well recognized mode collapse. We first proceed by mapping the desire…
Generative Adversarial Network