Not All Lotteries Are Made Equal
The Lottery Ticket Hypothesis (LTH) states that for a reasonably sized neural network, a sub-network within the same network yields no less performance than the dense counterpart when trained from the same initialization. This work investigates the relation between model size and the ease of finding these sparse sub-networks. We show through experiments that, surprisingly, under a finite budget, smaller models benefit more from Ticket Search (TS).
Code (0)
등록된 구현이 없습니다.
Tasks
AllRelationTicket SearchSimilar Papers 제목 키워드 기반
Making school choice lotteries transparent
Lotteries are commonly employed in school choice to fairly resolve priority ties; however, current practices leave students uninformed about their lottery outcomes when submitting preferences. This paper advocates for re…
Decision MakingRegret theory, Allais' Paradox, and Savage's omelet
We study a sufficiently general regret criterion for choosing between two probabilistic lotteries. For independent lotteries, the criterion is consistent with stochastic dominance and can be made transitive by a unique c…
Not All Lotteries Are Made Equal
The Lottery Ticket Hypothesis (LTH) states that for a reasonably sized neural network, there exists a subnetwork within the same network that yields no less performance than the dense counterpart when trained from the sa…
AllThe Near Miss Effect and the Framing of Lotteries
We present a framework for analyzing the near miss effect in lotteries. A decision maker (DM) facing a lottery, falsely interprets losing outcomes that are close to winning ones, as a sign that success is within reach. A…
Robust AI Evaluation through Maximal Lotteries
The standard way to evaluate language models on subjective tasks is through pairwise comparisons: an annotator chooses the "better" of two responses to a prompt. Leaderboards aggregate these comparisons into a single Bra…