paper-with-me

Papers

The Necessity of AI Audit Standards Boards

2024-04-11 · David Manheim, Sammy Martin, Mark Bailey, Mikhail Samin, Ross Greutzmacher

Auditing of AI systems is a promising way to understand and manage ethical problems and societal risks associated with contemporary AI systems, as well as some anticipated future risks. Efforts to develop standards for auditing Artificial Intelligence (AI) systems have therefore understandably gained momentum. However, we argue that creating auditing standards is not just insufficient, but actively harmful by proliferating unheeded and inconsistent standards, especially in light of the rapid evolution and ethical and safety challenges of AI. Instead, the paper proposes the establishment of an AI Audit Standards Board, responsible for developing and updating auditing methods and standards in line with the evolving nature of AI technologies. Such a body would ensure that auditing practices remain relevant, robust, and responsive to the rapid advancements in AI. The paper argues that such a governance structure would also be helpful for maintaining public trust in AI and for promoting a culture of safety and ethical responsibility within the AI industry. Throughout the paper, we draw parallels with other industries, including safety-critical industries like aviation and nuclear energy, as well as more prosaic ones such as financial accounting and pharmaceuticals. AI auditing should emulate those fields, and extend beyond technical assessments to include ethical considerations and stakeholder engagement, but we explain that this is not enough; emulating other fields' governance mechanisms for these processes, and for audit standards creation, is a necessity. We also emphasize the importance of auditing the entire development process of AI systems, not just the final products...

📄 PDF Abstract BibTeX arXiv:2404.13060

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Compliant But Unsatisfactory: The Gap Between Auditing Standards and Practices for Probabilistic Genotyping Software

2026-04-13 · Angela Jin, Alexander Asemota, Dan E. Krane, Nathaniel D. Adams 외 arxiv

AI governance efforts increasingly rely on audit standards: agreed-upon practices for conducting audits. However, poorly designed standards can hide and lend credibility to inadequate systems. We explore how an audit sta…

A Unified Perturbation Framework for Analyzing Leaderboard Stability and Manipulation

2026-05-15 · Hosna Oyarhoseini, Jimmy Lin, Amir-Hossein Karimi arxiv

Evaluation leaderboards such as LMArena play a central role in benchmarking large language models by aggregating pairwise human preferences into model rankings, yet the robustness of these rankings remains poorly underst…

AuditNet: A Conversational AI-based Security Assistant [DEMO]

2024-07-19 · Shohreh Deldari, Mohammad Goudarzi, Aditya Joshi, Arash Shaghaghi 외

In the age of information overload, professionals across various fields face the challenge of navigating vast amounts of documentation and ever-evolving standards. Ensuring compliance with standards, regulations, and con…

RetrievalRetrieval-augmented Generation

AgentAtlas: Beyond Outcome Leaderboards for LLM Agents

2026-05-19 · Parsa Mazaheri, Kasra Mazaheri arxiv

Large language model agents now act on codebases, browsers, operating systems, calendars, files, and tool ecosystems, but their evaluations often collapse behavior into final task success. AgentAtlas reframes agent evalu…

Automated Population-Level Audit Assurance via AI-Based Document Intelligence

2026-05-05 · Santosh Vasudevan, Velu Natarajan arxiv

Audit transaction testing validates accuracy and completeness of customer-facing statements against internal systems of record. Traditional manual, sample-based review of unstructured PDF statements is labor-intensive an…

Document AI