Generative Artificial Intelligence-Supported Pentesting: A Comparison between Claude Opus, GPT-4, and Copilot

2025-01-12 · Antonio López Martínez, Alejandro Cano, Antonio Ruiz-Martínez

The advent of Generative Artificial Intelligence (GenAI) has brought a significant change to our society. GenAI can be applied across numerous fields, with particular relevance in cybersecurity. Among the various areas of application, its use in penetration testing (pentesting) or ethical hacking processes is of special interest. In this paper, we have analyzed the potential of leading generic-purpose GenAI tools-Claude Opus, GPT-4 from ChatGPT, and Copilot-in augmenting the penetration testing process as defined by the Penetration Testing Execution Standard (PTES). Our analysis involved evaluating each tool across all PTES phases within a controlled virtualized environment. The findings reveal that, while these tools cannot fully automate the pentesting process, they provide substantial support by enhancing efficiency and effectiveness in specific tasks. Notably, all tools demonstrated utility; however, Claude Opus consistently outperformed the others in our experimental scenarios.

📄 PDF Abstract BibTeX arXiv:2501.06963

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Adam 설명 없음

Residual Connection 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Attention 설명 없음

Autonomous Penetration Testing using Reinforcement Learning

2019-05-15 · Jonathon Schwartz, Hanna Kurniawati

Penetration testing (pentesting) involves performing a controlled attack on a computer system in order to assess it's security. Although an effective method for testing security, pentesting requires highly skilled practi…

Q-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World

2026-05-11 · Pedro Conde, Henrique Branquinho, Valerio Mazzone, Bruno Mendes 외 arxiv

AI pentesting agents are increasingly credible as offensive security systems, but current benchmarks still provide limited guidance on which will perform best in real-world targets. Existing evaluation protocols assess a…

THEaiTRE: Artificial Intelligence to Write a Theatre Play

2020-06-25 · Rudolf Rosa, Ondřej Dušek, Tom Kocmi, David Mareček 외

We present THEaiTRE, a starting project aimed at automatic generation of theatre play scripts. This paper reviews related work and drafts an approach we intend to follow. We plan to adopt generative neural language model…

Machine Translation Translation

Use of What-if Scenarios to Help Explain Artificial Intelligence Models for Neonatal Health

2024-10-12 · Abdullah Mamun, Lawrence D. Devoe, Mark I. Evans, David W. Britt 외

Early detection of intrapartum risk enables interventions to potentially prevent or mitigate adverse labor outcomes such as cerebral palsy. Currently, there is no accurate automated system to predict such events to assis…

counterfactual Data Augmentation

The Opaque Law of Artificial Intelligence

2023-10-19 · Vincenzo Calderonio

The purpose of this paper is to analyse the opacity of algorithms, contextualized in the open debate on responsibility for artificial intelligence causation; with an experimental approach by which, applying the proposed …