Reading Between the Waves: Robust Topic Segmentation Using Inter-Sentence Audio Features
Spoken content, such as online videos and podcasts, often spans multiple topics, which makes automatic topic segmentation essential for user navigation and downstream applications. However, current methods do not fully leverage acoustic features, leaving room for improvement. We propose a multi-modal approach that fine-tunes both a text encoder and a Siamese audio encoder, capturing acoustic cues around sentence boundaries. Experiments on a large-scale dataset of YouTube videos show substantial gains over text-only and multi-modal baselines. Our model also proves more resilient to ASR noise and outperforms a larger text-only baseline on three additional datasets in Portuguese, German, and English, underscoring the value of learned acoustic features for robust topic segmentation.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Periodic recurrent waves of Covid-19 epidemics and vaccination campaign
While understanding of periodic recurrent waves of Covid-19 epidemics would aid to combat the pandemics, quantitative analysis of data over a two years period from the outbreak, is lacking. The complexity of Covid-19 rec…
SECTOR: A Neural Model for Coherent Topic Segmentation and Classification
When searching for information, a human reader first glances over a document, spots relevant sections and then focuses on a few sentences for resolving her intention. However, the high variance of document structure comp…
ClassificationGeneral ClassificationReading ComprehensionSegmentationDeep learning for COVID-19 topic modelling via Twitter: Alpha, Delta and Omicron
Topic modelling with innovative deep learning methods has gained interest for a wide range of applications that includes COVID-19. Topic modelling can provide, psychological, social and cultural insights for understandin…
ManagementUncovering potential effects of spontaneous waves on synaptic development: the visual system as a model
Spontaneous waves are ubiquitous during early brain development and are hypothesized to drive the development of receptive fields (RFs). Different stages of spontaneous waves in the retina have been observed to coincide …
TripletLearning Common Harmonic Waves on Stiefel Manifold -- A New Mathematical Approach for Brain Network Analyses
Converging evidence shows that disease-relevant brain alterations do not appear in random brain locations, instead, its spatial pattern follows large scale brain networks. In this context, a powerful network analysis app…