PPOM: Marginalizing Patch-Grid Phase for CLIP-Based Generalizable Vision-Language Prompt Tuning
Prompt tuning adapts CLIP-based vision-language models with few trainable parameters, yet its predictions remain sensitive to the spatial sampling imposed by a frozen vision transformer. In particular, non-overlapping patch tokenization makes predictions depend on the alignment (phase) between image and the patch lattice. To reduce prediction sensitivity to patch-grid alignment, we introduce Patch-Phase Orbit Marginalization (PPOM), a training-free inference operator that treats phase shift as a nuisance variable. Given a patch stride, PPOM evaluates the identity view and reflection-padded translations, pairs opposite shifts into horizontal, vertical, and diagonal antithetic families, and assigns equal mass to these families and the identity prediction to avoid view-count bias during phase integration. In summary, PPOM provides a deterministic interface between prompt adaptation and patch-grid sensitivity. Across multiple prompt-learning hosts, PPOM improves host performance without re-training.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Phase Marginalization for Patch-Grid Instability in Vision Transformers
Vision Transformers operate on fixed patch grids, which can introduce phase-dependent instability for dense prediction: changing the patch partition can change the token evidence available to a pixel, especially near bou…
Optimal Grid-Forming Control of Battery Energy Storage Systems Providing Multiple Services: Modelling and Experimental Validation
This paper proposes and experimentally validates a joint control and scheduling framework for a grid-forming converter-interfaced BESS providing multiple services to the electrical grid. The framework is designed to disp…
Model Predictive ControlSchedulingPhysics-Aware Heterogeneous GNN Architecture for Real-Time BESS Optimization in Unbalanced Distribution Systems
Battery energy storage systems (BESS) have become increasingly vital in three-phase unbalanced distribution grids for maintaining voltage stability and enabling optimal dispatch. However, existing deep learning approache…
InfraQR: Edge-Placed QR-Inspired Structured Patch Attacks on Infrared Vision-Language Models
Infrared vision-language models are increasingly used for perception under low-light and adverse visual conditions, yet their robustness to localized structured perturbations remains underexplored. Existing infrared adve…
Visual Question AnsweringANOCA: AC Network-aware Optimal Curtailment Approach for Dynamic Hosting Capacity
With exponential growth in distributed energy resources (DERs) coupled with at-capacity distribution grid infrastructure, prosumers cannot always export all extra power to the grid without violating technical limits. Con…