FBK's Long-form SpeechLLMs for IWSLT 2026 Instruction Following
This paper describes our submission to the IWSLT 2026 Instruction Following shared task. SpeechLLMs are developed for both short-form and long-form speech instruction following under constrained settings. For the short track, strong performance is achieved on MCIF, with a SIFS score of 2.0708. For the long track, three speech segmentation methods are explored, and the HIFS score is introduced to account for unstable long-form generation. Experimental results show that fixed 30-second segmentation provides the most robust long-form performance, achieving the highest HIFS score of 2.0663. Further analysis shows that hallucination mainly manifests as repetitive insertions in generated outputs, substantially affecting ASR and SSUM, while short-form capabilities are largely retained after long-form extension.
Code (0)
등록된 구현이 없습니다.
Tasks
Instruction FollowingSimilar Papers 제목 키워드 기반
Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026
With the advent of Large Language Models, single-task and token-based multi-task models have evolved into instruction-based systems that infer task and target language implicitly from natural language prompts. This trend…
Instruction FollowingData AugmentationNAVER LABS System Re-implementation for the IWSLT 2026 Instruction-Following Task
We re-implement the NAVER LABS IWSLT 2025 instruction-following pipeline for the IWSLT 2026 Shared Task (constrained condition, short audio track), adapting it to the mandated components: SeamlessM4T-v2-large as the spee…
SpeechLLMs for Large-scale Contextualized Zero-shot Slot Filling
Slot filling is a crucial subtask in spoken language understanding (SLU), traditionally implemented as a cascade of speech recognition followed by one or more natural language understanding (NLU) components. The recent a…
Natural Language UnderstandingSpoken Language UnderstandingZero-shot Slot FillingSpeech RecognitionKIT's Offline Speech Translation and Instruction Following Submission for IWSLT 2025
The scope of the International Workshop on Spoken Language Translation (IWSLT) has recently broadened beyond traditional Speech Translation (ST) to encompass a wider array of tasks, including Speech Question Answering an…
Automatic Speech RecognitionInstruction FollowingQuestion Answeringspeech-recognition+2InSerter: Speech Instruction Following with Unsupervised Interleaved Pre-training
Recent advancements in speech large language models (SpeechLLMs) have attracted considerable attention. Nonetheless, current methods exhibit suboptimal performance in adhering to speech instructions. Notably, the intelli…
Instruction Followingtext-to-speechText to Speech