paper-with-me

홈 › Papers

Dual Teacher Knowledge Distillation with Domain Alignment for Face Anti-spoofing

2024-01-02 · Zhe Kong, Wentian Zhang, Tao Wang, Kaihao Zhang, Yuexiang Li, Xiaoying Tang, Wenhan Luo

Face recognition systems have raised concerns due to their vulnerability to different presentation attacks, and system security has become an increasingly critical concern. Although many face anti-spoofing (FAS) methods perform well in intra-dataset scenarios, their generalization remains a challenge. To address this issue, some methods adopt domain adversarial training (DAT) to extract domain-invariant features. However, the competition between the encoder and the domain discriminator can cause the network to be difficult to train and converge. In this paper, we propose a domain adversarial attack (DAA) method to mitigate the training instability problem by adding perturbations to the input images, which makes them indistinguishable across domains and enables domain alignment. Moreover, since models trained on limited data and types of attacks cannot generalize well to unknown attacks, we propose a dual perceptual and generative knowledge distillation framework for face anti-spoofing that utilizes pre-trained face-related models containing rich face priors. Specifically, we adopt two different face-related models as teachers to transfer knowledge to the target student model. The pre-trained teacher models are not from the task of face anti-spoofing but from perceptual and generative tasks, respectively, which implicitly augment the data. By combining both DAA and dual-teacher knowledge distillation, we develop a dual teacher knowledge distillation with domain alignment framework (DTDA) for face anti-spoofing. The advantage of our proposed method has been verified through extensive ablation studies and comparison with state-of-the-art methods on public datasets across multiple protocols.

📄 PDF Abstract BibTeX arXiv:2401.01102

Code (0)

등록된 구현이 없습니다.

Tasks

Adversarial AttackFace Anti-SpoofingFace RecognitionKnowledge Distillation

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Similar Papers 제목 키워드 기반

DWA-KD: Dual-Space Weighting and Time-Warped Alignment for Cross-Tokenizer Knowledge Distillation

2026-02-25 · Duc Trung Vu, Pham Khanh Chi, Dat Phi Van, Linh Ngo Van 외 arxiv

Knowledge Distillation (KD) has emerged as a crucial technique for compressing Large Language Models (LLMs). Although existing cross-tokenizer KD methods have made notable progress, their effectiveness remains constraine…

Knowledge Distillation

Dual-Teacher De-biasing Distillation Framework for Multi-domain Fake News Detection

2023-12-02 · Jiayang Li, Xuan Feng, Tianlong Gu, Liang Chang

Multi-domain fake news detection aims to identify whether various news from different domains is real or fake and has become urgent and important. However, existing methods are dedicated to improving the overall performa…

Fake News DetectionKnowledge Distillation

Advantage-Guided Distillation for Preference Alignment in Small Language Models

2025-02-25 · Shiping Gao, Fanqi Wan, Jiajian Guo, Xiaojun Quan 외

Alignment techniques enable Large Language Models (LLMs) to generate outputs that align with human preferences and play a crucial role in their effectiveness. However, their impact often diminishes when applied to Small …

Knowledge Distillation

Distilled Circuits: A Mechanistic Study of Internal Restructuring in Knowledge Distillation

2025-05-16 · Reilly Haskins, Benjamin Adams

Knowledge distillation compresses a larger neural model (teacher) into smaller, faster student models by training them to match teacher outputs. However, the internal computational transformations that occur during this …

Knowledge Distillation

CoT2Align: Cross-Chain of Thought Distillation via Optimal Transport Alignment for Language Models with Different Tokenizers

2025-02-24 · Anh Duc Le, Tu Vu, Nam Le Hai, Nguyen Thi Ngoc Diep 외

Large Language Models (LLMs) achieve state-of-the-art performance across various NLP tasks but face deployment challenges due to high computational costs and memory constraints. Knowledge distillation (KD) is a promising…

Knowledge Distillation