ForgettingOT: Certified Speculative Batching from Sinkhorn's Projective Forgetting
Positive two-marginal entropic optimal transport is solved by a nonlinear, positive, order-preserving, homogeneous Sinkhorn map. After quotienting the dual scaling gauge, we show that the active eigenmode of the fixed-point Jacobian $J_t^\star=QP=P^\star P$, generically $λ_2(QP)$, controls the strict correction tail. The projective-residual ratio converges to this mode, and the additional certified cycles required for tolerance $θ_τ$ scale as $\log(ρ/θ_τ)/[-\logλ_2]+O(1)$. ForgettingOT turns this nonlinear Perron--Frobenius fact into a certified executor for streams of related Sinkhorn problems. A computable projective variation $Ω_t$ in the marginals and kernel bounds the carry residual, while a verified contraction $q_t$ gives candidate repair depth. A window theorem converts these depths and the audit grid into bounds on packed work, collective rounds, overshoot, and fallback. Empirical tail estimates allocate work but never authorize release; current-instance certificates or measured marginal residuals do so, with ordinary Sinkhorn as fallback. On 15 FP64 A100/OTT-JAX cells, the observed quotient slow-mode ratio agrees with $λ_2(QP)$ to $9.84\times10^{-6}$. On controlled four-A100 streams, the complete executor is $1.42\times$--$3.55\times$ faster than sequential soft $c$-transform warm starts, with 30/30 paired wins and no violations of the $10^{-3}$ marginal tolerance. Eight-A100 support-4096 streams give $2.584\times$--$2.945\times$ wall-time speedup and $4.285\times$--$4.615\times$ fewer vector-collective rounds. The outer executor composes with target-preserving Sinkhorn accelerators; a changed map or approximate target needs a contraction or error bridge before inheriting the repair-depth bound.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Hilbert's projective metric for functions of bounded growth and exponential convergence of Sinkhorn's algorithm
Motivated by the entropic optimal transport problem in unbounded settings, we study versions of Hilbert's projective metric for spaces of integrable functions of bounded growth. These versions of Hilbert's metric origina…
The Synergy of Speculative Decoding and Batching in Serving Large Language Models
Large Language Models (LLMs) like GPT are state-of-the-art text generation models that provide significant assistance in daily routines. However, LLM execution is inherently sequential, since they only produce one token …
GPUText GenerationXShare: Collaborative in-Batch Expert Sharing for Faster MoE Inference
Mixture-of-Experts (MoE) architectures are increasingly used to efficiently scale large language models. However, in production inference, request batching and speculative decoding significantly amplify expert activation…
Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction
Large language models (LLMs) have been driving a new wave of interactive AI applications across numerous domains. However, efficiently serving LLM inference requests is challenging due to their unpredictable execution ti…
BlockingManagementSchedulingSSSD: Simply-Scalable Speculative Decoding
Over the past year, Speculative Decoding has gained popularity as a technique for accelerating Large Language Model inference. While several methods have been introduced, most struggle to deliver satisfactory performance…
Language ModelingLanguage ModellingLarge Language Model