An Explainable Proxy Model for Multiabel Audio Segmentation
Audio signal segmentation is a key task for automatic audio indexing. It consists of detecting the boundaries of class-homogeneous segments in the signal. In many applications, explainable AI is a vital process for transparency of decision-making with machine learning. In this paper, we propose an explainable multilabel segmentation model that solves speech activity (SAD), music (MD), noise (ND), and overlapped speech detection (OSD) simultaneously. This proxy uses the non-negative matrix factorization (NMF) to map the embedding used for the segmentation to the frequency domain. Experiments conducted on two datasets show similar performances as the pre-trained black box model while showing strong explainability features. Specifically, the frequency bins used for the decision can be easily identified at both the segment level (local explanations) and global level (class prototypes).
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingSegmentationSimilar Papers 제목 키워드 기반
Explainable by-design Audio Segmentation through Non-Negative Matrix Factorization and Probing
Audio segmentation is a key task for many speech technologies, most of which are based on neural networks, usually considered as black boxes, with high-level performances. However, in many domains, among which health or …
InformativenessSegmentationSegmenting Subtitles for Correcting ASR Segmentation Errors
Typical ASR systems segment the input audio into utterances using purely acoustic information, which may not resemble the sentence-like units that are expected by conventional machine translation (MT) systems for Spoken …
Information RetrievalMachine TranslationRetrievalSegmentation+2OpenAVS: Training-Free Open-Vocabulary Audio Visual Segmentation with Foundational Models
Audio-visual segmentation aims to separate sounding objects from videos by predicting pixel-level masks based on audio signals. Existing methods primarily concentrate on closed-set scenarios and direct audio-visual align…
Pseudo LabelSemantic SegmentationTransfer LearningGenerating In-Distribution Proxy Graphs for Explaining Graph Neural Networks
Graph Neural Networks (GNNs) have become a building block in graph data processing, with wide applications in critical domains. The growing needs to deploy GNNs in high-stakes applications necessitate explainability for …
Decision MakingRelational Proxy Loss for Audio-Text based Keyword Spotting
In recent years, there has been an increasing focus on user convenience, leading to increased interest in text-based keyword enrollment systems for keyword spotting (KWS). Since the system utilizes text input during the …
Keyword SpottingMetric LearningTriplet