Low impact agency: review and discussion
Powerful artificial intelligence poses an existential threat if the AI decides to drastically change the world in pursuit of its goals. The hope of low-impact artificial intelligence is to incentivize AI to not do that just because this causes a large impact in the world. In this work, we first review the concept of low-impact agency and previous proposals to approach the problem, and then propose future research directions in the topic, with the goal to ensure low-impactedness is useful in making AI safe.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Generative AI and Agency in Education: A Critical Scoping Review and Thematic Analysis
This scoping review examines the relationship between Generative AI (GenAI) and agency in education, analyzing the literature available through the lens of Critical Digital Pedagogy. Following PRISMA-ScR guidelines, we c…
Teacher agency in the age of generative AI: towards a framework of hybrid intelligence for learning design
Generative AI (genAI) is being used in education for different purposes. From the teachers' perspective, genAI can support activities such as learning design. However, there is a need to study the impact of genAI on the …
Prompt Injection Attacks on LLM Generated Reviews of Scientific Publications
The ongoing intense discussion on rising LLM usage in the scientific peer-review process has recently been mingled by reports of authors using hidden prompt injections to manipulate review scores. Since the existence of …
A Large Scale Randomized Controlled Trial on Herding in Peer-Review Discussions
Peer review is the backbone of academia and humans constitute a cornerstone of this process, being responsible for reviewing papers and making the final acceptance/rejection decisions. Given that human decision making is…
Decision MakingWhat happens when reviewers receive AI feedback in their reviews?
AI is reshaping academic research, yet its role in peer review remains polarising and contentious. Advocates see its potential to reduce reviewer burden and improve quality, while critics warn of risks to fairness, accou…