paper-with-me

홈 › Papers

Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability

2026-07-11 · Lingwei Wei, Dou Hu, Wei Zhou, Songlin Hu, Philip S. Yu arxiv

Large language models (LLMs) have transformed misinformation from a primarily content-centric problem into a broader ecosystem-level security challenge. When misused, LLMs create risks beyond false content generation, enabling attacks on the social contexts, evidence sources, retrieval corpora, and verification workflows that misinformation defense depends on. In this paper, we introduce a role-layer framework to unify these risks and defenses. The role dimension characterizes LLMs as attackers, defenders, and vulnerable components of verification systems, while the layer dimension covers content, social contexts, evidence environments, and verification workflows. Guided by this framework, we organize LLM-enabled attacks, investigate LLM-based detection and verification methods, analyze vulnerabilities in LLM-centric detection paradigms, and discuss existing countermeasures against LLM-enabled attacks. Building on this synthesis, we identify three key open challenges: moving from static detection accuracy to budgeted ecosystem-level risk evaluation, hardening LLM-centered verification pipelines against adversarial manipulation, and deploying auditable human-in-the-loop verification systems for trustworthy real-world misinformation defense.

📄 PDF Abstract BibTeX arXiv:2607.10402

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

On the Risk of Misinformation Pollution with Large Language Models

2023-05-23 · Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov 외

In this paper, we comprehensively investigate the potential misuse of modern Large Language Models (LLMs) for generating credible-sounding misinformation and its subsequent impact on information-intensive applications, p…

MisinformationOpen-Domain Question AnsweringQuestion Answering

Preventing Jailbreak Prompts as Malicious Tools for Cybercriminals: A Cyber Defense Perspective

2024-11-25 · Jean Marie Tshimula, Xavier Ndona, D'Jeff K. Nkashama, Pierre-Martin Tardif 외

Jailbreak prompts pose a significant threat in AI and cybersecurity, as they are crafted to bypass ethical safeguards in large language models, potentially enabling misuse by cybercriminals. This paper analyzes jailbreak…

Misinformation

A Survey on Proactive Defense Strategies Against Misinformation in Large Language Models

2025-07-05 · Shuliang Liu, Hongyi Liu, Aiwei Liu, Bingchen Duan 외

The widespread deployment of large language models (LLMs) across critical domains has amplified the societal risks posed by algorithmically generated misinformation. Unlike traditional false content, LLM-generated misinf…

Misinformation

Fighting Fire with Fire: Adversarial Prompting to Generate a Misinformation Detection Dataset

2024-01-09 · Shrey Satapara, Parth Mehta, Debasis Ganguly, Sandip Modha

The recent success in language generation capabilities of large language models (LLMs), such as GPT, Bard, Llama etc., can potentially lead to concerns about their possible misuse in inducing mass agitation and communal …

MisinformationText Generation

Benchmarking Misuse Mitigation Against Covert Adversaries

2025-06-06 · Davis Brown, Mahdi Sabbaghi, Luze Sun, Alexander Robey 외

Existing language model safety evaluations focus on overt attacks and low-stakes tasks. Realistic attackers can subvert current safeguards by requesting help on small, benign-seeming tasks across many independent queries…

BenchmarkingLanguage ModelingLanguage Modelling