CIRCUS: Circuit Consensus under Uncertainty via Stability Ensembles
Every mechanistic circuit carries an invisible asterisk: it reflects not just the model's computation, but the analyst's choice of pruning threshold. Change that choice and the circuit changes, yet current practice treats a single pruned subgraph as ground truth with no way to distinguish robust structure from threshold artifacts. We introduce CIRCUS, which reframes circuit discovery as a problem of uncertainty over explanations. CIRCUS prunes one attribution graph under B configurations, assigns each edge an empirical inclusion frequency s(e) in [0,1] measuring how robustly it survives across the configuration family, and extracts a consensus circuit of edges present in every view. This yields a principled core/contingent/noise decomposition (analogous to posterior model-inclusion indicators in Bayesian variable selection) that separates robust structure from threshold-sensitive artifacts, with negligible overhead. On Gemma-2-2B and Llama-3.2-1B, consensus circuits are 40x smaller than the union of all configurations while retaining comparable influence-flow explanatory power, consistently outperform influence-ranked and random baselines, and are confirmed causally relevant by activation patching.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Robust Output Feedback Consensus for Networked Identical Nonlinear Negative-Imaginary Systems
A robust output feedback consensus problem for networked identical nonlinear negative-imaginary (NI) systems is investigated in this paper. Output consensus is achieved by applying identical linear output strictly negati…
Stabilizing Transformer Training Through Consensus
Standard attention-based transformers are known to exhibit instability under learning rate overspecification during training, particularly at high learning rates. While various methods have been proposed to improve resil…
What If Consensus Lies? Selective-Complementary Reinforcement Learning at Test Time
Test-Time Reinforcement Learning (TTRL) enables Large Language Models (LLMs) to enhance reasoning capabilities on unlabeled test streams by deriving pseudo-rewards from majority voting consensus. However, existing TTRL m…
Reinforcement LearningNN-ETM: Enabling safe neural network-based event-triggering mechanisms for consensus problems
Event-triggering mechanisms (ETM) have been developed for consensus problems to reduce communication while ensuring performance guarantees, but their design has grown increasingly complex by incorporating the agent's loc…
Impossibility results on stability of phylogenetic consensus methods
We answer two questions raised by Bryant, Francis and Steel in their work on consensus methods in phylogenetics. Consensus methods apply to every practical instance where it is desired to aggregate a set of given phyloge…