Superintelligence cannot be contained: Lessons from Computability Theory
Superintelligence is a hypothetical agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. In light of recent advances in machine intelligence, a number of scientists, philosophers and technologists have revived the discussion about the potential catastrophic risks entailed by such an entity. In this article, we trace the origins and development of the neo-fear of superintelligence, and some of the major proposals for its containment. We argue that such containment is, in principle, impossible, due to fundamental limits inherent to computing itself. Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world, strict containment requires simulations of such a program, something theoretically (and practically) infeasible.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
On Formally Undecidable Traits of Intelligent Machines
Building on work by Alfonseca et al. (2021), we study the conditions necessary for it to be logically possible to prove that an arbitrary artificially intelligent machine will exhibit certain behavior. To do this, we dev…
Hallucinations are inevitable but can be made statistically negligible. The "innate" inevitability of hallucinations cannot explain practical LLM issues
Hallucinations, a phenomenon where a language model (LM) generates nonfactual content, pose a significant challenge to the practical deployment of LMs. While many empirical methods have been proposed to mitigate hallucin…
Language ModelingLanguage ModellingExtending and Automating Basic Probability Theory with Propositional Computability Logic
Classical probability theory is formulated using sets. In this paper, we extend classical probability theory with propositional computability logic. Unlike other formalisms, computability logic is built on the notion of …
Aligning Artificial Superintelligence via a Multi-Box Protocol
We propose a novel protocol for aligning artificial superintelligence (ASI) based on mutual verification among multiple isolated systems that self-modify to achieve alignment. The protocol operates by containing multiple…
How To Solve Moral Conundrums with Computability Theory
Various moral conundrums plague population ethics: the Non-Identity Problem, the Procreation Asymmetry, the Repugnant Conclusion, and more. I argue that the aforementioned moral conundrums have a structure neatly account…
EthicsPhilosophy