Reference Traces for Auditing Invisible Weight Updates and Guiding Exact-Budget Protection
Direct low-precision write-back can erase optimizer proposals, while aggregate update visibility need not identify parameters worth protecting. We study two uses of high-precision reference traces: candidate-matched auditing before a low-precision run and regime-matched allocation before unseen targets. The exact target-code event is distinct from aggregate proxies, which are not pathwise certificates. In a controlled two-layer grid, an archived proxy tracks crossings spanning 384x in time, with median predicted/measured ratio 1.000 and 94.5% within 15%. In analytic-grid GPT-2-124M/OpenWebText runs, changing only post-optimizer write-back from round-to-nearest to stochastic rounding recovers 89.2-92.2% of the loss gap through 10,000 steps. The effect repeats in a separate modern-decoder bundle, but its visibility trajectories are uninformative. In a fully prospective 162.2M-parameter allocation test, a 600-step source trace freezes a 10% fp32-master tile mask before three unseen targets. It recovers 60.4-61.0% (mean 60.7%), versus 60.5-61.5% (mean 60.8%) for target-specific masks and 17.5-18.2% for the best fixed composition-matched controls; freeze-only recovers 0.7-0.9%. A separately frozen 197.6M-parameter depth migration recovers 65.9-67.2% (mean 66.4%). A matched systems check reduces PyTorch peak allocation by 584.65 MB versus a blanket master, but adds 65.85 MB and has 7.196% lower arm-median timed-window throughput versus direct round-to-nearest, with no validated break-even. A post hoc same-grid ECO-Adam comparator recovers 96.4-96.6%, versus 60.4-61.0% for the source mask; its systems cost is unmeasured. Thus reference traces can audit aggregate visibility and separately guide exact-budget protection, but the audit did not transfer diagnostically and the current allocator has no net systems benefit.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Beyond Passive Aggregation: Active Auditing and Topology-Aware Defense in Decentralized Federated Learning
Decentralized Federated Learning (DFL) remains highly vulnerable to adaptive backdoor attacks designed to bypass traditional passive defense metrics. To address this limitation, we shift the defensive paradigm toward a n…
Federated LearningPredictive Auditing of Hidden Tokens in LLM APIs via Reasoning Length Estimation
Commercial LLM services often conceal internal reasoning traces while still charging users for every generated token, including those from hidden intermediate steps, raising concerns of token inflation and potential over…
TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modeling
LLM agents increasingly operate through multi-turn tool use and environment interaction, where safety risks often emerge from intermediate steps long before they surface in the final outcome. Reactive auditing is therefo…
CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIs
As post-training techniques evolve, large language models (LLMs) are increasingly augmented with structured multi-step reasoning abilities, often optimized through reinforcement learning. These reasoning-enhanced models …
Auditing Fairness under Model Updates: Fundamental Complexity and Property-Preserving Updates
As machine learning models become increasingly embedded in societal infrastructure, auditing them for bias is of growing importance. However, in real-world deployments, auditing is complicated by the fact that model owne…