Prompting with Sign Parameters for Low-resource Sign Language Instruction Generation
Sign Language (SL) enables two-way communication for the deaf and hard-of-hearing community, yet many sign languages remain under-resourced in the AI space. Sign Language Instruction Generation (SLIG) produces step-by-step textual instructions that enable non-SL users to imitate and learn SL gestures, promoting two-way interaction. We introduce BdSLIG, the first Bengali SLIG dataset, used to evaluate Vision Language Models (VLMs) (i) on under-resourced SLIG tasks, and (ii) on long-tail visual concepts, as Bengali SL is unlikely to appear in the VLM pre-training data. To enhance zero-shot performance, we introduce Sign Parameter-Infused (SPI) prompting, which integrates standard SL parameters, like hand shape, motion, and orientation, directly into the textual prompts. Subsuming standard sign parameters into the prompt makes the instructions more structured and reproducible than free-form natural text from vanilla prompting. We envision that our work would promote inclusivity and advancement in SL learning systems for the under-resourced communities.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Effectively Prompting Small-sized Language Models for Cross-lingual Tasks via Winning Tickets
Current soft prompt methods yield limited performance when applied to small-sized models (fewer than a billion parameters). Deep prompt-tuning, which entails prepending parameters in each layer for enhanced efficacy, pre…
Language ModelingLanguage ModellingMasked Language ModelingPrompt LearningPrompting and Adapter Tuning for Self-supervised Encoder-Decoder Speech Model
Prompting and adapter tuning have emerged as efficient alternatives to fine-tuning (FT) methods. However, existing studies on speech prompting focused on classification tasks and failed on more complex sequence generatio…
Cross-Lingual ASRDecoderslot-fillingSlot FillingTrace-of-Thought Prompting: Investigating Prompt-Based Knowledge Distillation Through Question Decomposition
Knowledge distillation allows smaller neural networks to emulate the performance of larger, teacher models with reduced computational demands. Traditional methods for Large Language Models (LLMs) often necessitate extens…
GSM8KKnowledge DistillationMathProblem DecompositionCan open source large language models be used for tumor documentation in Germany? -- An evaluation on urological doctors' notes
Tumor documentation in Germany is largely done manually, requiring reading patient records and entering data into structured databases. Large language models (LLMs) could potentially enhance this process by improving eff…
Chain-of-Translation Prompting (CoTR): A Novel Prompting Technique for Low Resource Languages
This paper introduces Chain of Translation Prompting (CoTR), a novel strategy designed to enhance the performance of language models in low-resource languages. CoTR restructures prompts to first translate the input conte…
Hate Speech DetectionSentiment AnalysisSynthetic Data GenerationText Generation+1