홈 › Papers

Llama Guard 3-1B-INT4: Compact and Efficient Safeguard for Human-AI Conversations

2024-11-18 · Igor Fedorov, Kate Plawiak, Lemeng Wu, Tarek Elgamal, Naveen Suda, Eric Smith, Hongyuan Zhan, Jianfeng Chi, Yuriy Hulovatyy, Kimish Patel, Zechun Liu, Changsheng Zhao, Yangyang Shi, Tijmen Blankevoort, Mahesh Pasupuleti, Bilge Soran, Zacharie Delpierre Coudert, Rachad Alao, Raghuraman Krishnamoorthi, Vikas Chandra

This paper presents Llama Guard 3-1B-INT4, a compact and efficient Llama Guard model, which has been open-sourced to the community during Meta Connect 2024. We demonstrate that Llama Guard 3-1B-INT4 can be deployed on resource-constrained devices, achieving a throughput of at least 30 tokens per second and a time-to-first-token of 2.5 seconds or less on a commodity Android mobile CPU. Notably, our experiments show that Llama Guard 3-1B-INT4 attains comparable or superior safety moderation scores to its larger counterpart, Llama Guard 3-1B, despite being approximately 7 times smaller in size (440MB).

📄 PDF Abstract BibTeX arXiv:2411.17713

Code (0)

등록된 구현이 없습니다.

Tasks

CPU

Methods 이 논문이 사용한 방법론

LLaMA LLaMA is a collection of foundation language models ranging from 7B to 65B parameters. It is based on the transformer architecture with various improvements that were…

Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations

2024-11-15 · Jianfeng Chi, Ujjwal Karn, Hongyuan Zhan, Eric Smith 외

We introduce Llama Guard 3 Vision, a multimodal LLM-based safeguard for human-AI conversations that involves image understanding: it can be used to safeguard content for both multimodal LLM inputs (prompt classification)…

Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations

2023-12-07 · Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta 외

We introduce Llama Guard, an LLM-based input-output safeguard model geared towards Human-AI conversation use cases. Our model incorporates a safety risk taxonomy, a valuable tool for categorizing a specific set of safety…

Classification Language Modelling Multi-class Classification

AprielGuard

2025-12-23 · Jaykumar Kasundra, Anjaneya Praharaj, Sourabh Surana, Lakshmi Sirisha Chodisetty 외 arxiv

Safeguarding large language models (LLMs) against unsafe or adversarial behavior is critical as they are increasingly deployed in conversational and agentic settings. Existing moderation tools often treat safety risks (e…

Towards Healthy AI: Large Language Models Need Therapists Too

2023-04-02 · Baihan Lin, Djallel Bouneffouf, Guillermo Cecchi, Kush R. Varshney

Recent advances in large language models (LLMs) have led to the development of powerful AI chatbots capable of engaging in natural and human-like conversations. However, these chatbots can be potentially harmful, exhibit…

Chatbot

SEALGuard: Safeguarding the Multilingual Conversations in Southeast Asian Languages for LLM Software Systems

2025-07-11 · Wenliang Shan, Michael Fu, Rui Yang, Chakkrit Tantithamthavorn arxiv

Safety alignment is critical for LLM-powered systems. While recent LLM-powered guardrail approaches such as LlamaGuard achieve high detection accuracy of unsafe inputs written in English (e.g., ``How to create a bomb?'')…