Toward Idealized Decision Theory
This paper motivates the study of decision theory as necessary for aligning smarter-than-human artificial systems with human interests. We discuss the shortcomings of two standard formulations of decision theory, and demonstrate that they cannot be used to describe an idealized decision procedure suitable for approximation by artificial systems. We then explore the notions of policy selection and logical counterfactuals, two recent insights into decision theory that point the way toward promising paths for future research.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Dynamic Control for Random Access in Deadline-Constrained Broadcasting
This paper considers random access in deadline-constrained broadcasting with frame-synchronized traffic. To enhance the maximum achievable timely delivery ratio (TDR), we define a dynamic control scheme that allows each …
Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillation
We provide a general convergence theorem of an idealized stochastic Polyak step size called SPS$^*$. Besides convexity, we only assume a local expected gradient bound, that includes locally smooth and locally Lipschitz l…
A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation
Activation patching and weight-space ablation both claim a component is causally responsible for a behavior, yet they act on different objects: one forward pass versus the parameters behind every forward pass. We ask whe…
Towards a probability-free theory of continuous martingales
Without probability theory, we define classes of supermartingales, martingales, and semimartingales in idealized financial markets with continuous price paths. This allows us to establish probability-free versions of a n…
Optimal Teaching for Limited-Capacity Human Learners
Basic decisions, such as judging a person as a friend or foe, involve categorizing novel stimuli. Recent work finds that people’s category judgments are guided by a small set of examples that are retrieved from memory at…
Retrieval