TranssionADD: A multi-frame reinforcement based sequence tagging model for audio deepfake detection

2023-06-27 · Jie Liu, Zhiba Su, Hui Huang, Caiyan Wan, Quanxiu Wang, Jiangli Hong, Benlai Tang, Fengjie Zhu

Thanks to recent advancements in end-to-end speech modeling technology, it has become increasingly feasible to imitate and clone a user`s voice. This leads to a significant challenge in differentiating between authentic and fabricated audio segments. To address the issue of user voice abuse and misuse, the second Audio Deepfake Detection Challenge (ADD 2023) aims to detect and analyze deepfake speech utterances. Specifically, Track 2, named the Manipulation Region Location (RL), aims to pinpoint the location of manipulated regions in audio, which can be present in both real and generated audio segments. We propose our novel TranssionADD system as a solution to the challenging problem of model robustness and audio segment outliers in the trace competition. Our system provides three unique contributions: 1) we adapt sequence tagging task for audio deepfake detection; 2) we improve model generalization by various data augmentation techniques; 3) we incorporate multi-frame detection (MFD) module to overcome limited representation provided by a single frame and use isolated-frame penalty (IFP) loss to handle outliers in segments. Our best submission achieved 2nd place in Track 2, demonstrating the effectiveness and robustness of our proposed system.

📄 PDF Abstract BibTeX arXiv:2306.15212

Code (0)

등록된 구현이 없습니다.

Tasks

Audio Deepfake Detection Data Augmentation DeepFake Detection Face Swapping

A New Concept of Deep Reinforcement Learning based Augmented General Sequence Tagging System

2018-12-26 · Yu Wang, Abhishek Patel, Hongxia Jin

In this paper, a new deep reinforcement learning based augmented general sequence tagging system is proposed. The new system contains two parts: a deep neural network (DNN) based sequence tagging model and a deep reinfor…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A New Concept of Deep Reinforcement Learning based Augmented General Tagging System

2018-08-01 · COLING 2018 8 · Yu Wang, Abhishek Patel, Hongxia Jin

In this paper, a new deep reinforcement learning based augmented general tagging system is proposed. The new system contains two parts: a deep neural network (DNN) based sequence labeling model and a deep reinforcement l…

Deep Reinforcement Learning Named Entity Recognition (NER)reinforcement-learning Reinforcement Learning+3

A Tree Search Algorithm for Sequence Labeling

2018-04-29 · Yadi Lao, Jun Xu, Yanyan Lan, Jiafeng Guo 외

In this paper we propose a novel reinforcement learning based model for sequence tagging, referred to as MM-Tag. Inspired by the success and methodology of the AlphaGo Zero, MM-Tag formalizes the problem of sequence tagg…

Chunking Decision Making reinforcement-learning Reinforcement Learning+3

Deconstructing Supertagging into Multi-Task Sequence Prediction

2019-11-01 · CONLL 2019 11 · Zhenqi Zhu, Anoop Sarkar

Supertagging is a sequence prediction task where each word is assigned a piece of complex syntactic structure called a supertag. We provide a novel approach to multi-task learning for Tree Adjoining Grammar (TAG) superta…

Multi-Task Learning Prediction TAG

An Investigation towards Differentially Private Sequence Tagging in a Federated Framework

2021-06-01 · NAACL (PrivateNLP) 2021 6 · Abhik Jana, Chris Biemann

To build machine learning-based applications for sensitive domains like medical, legal, etc. where the digitized text contains private information, anonymization of text is required for preserving privacy. Sequence taggi…

Federated Learning named-entity-recognition Named Entity Recognition Named Entity Recognition (NER)+2