Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home
Enriching the quality of early childhood education with interactive math learning at home systems, empowered by recent advances in conversational AI technologies, is slowly becoming a reality. With this motivation, we implement a multimodal dialogue system to support play-based learning experiences at home, guiding kids to master basic math concepts. This work explores Spoken Language Understanding (SLU) pipeline within a task-oriented dialogue system developed for Kid Space, with cascading Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) components evaluated on our home deployment data with kids going through gamified math learning activities. We validate the advantages of a multi-task architecture for NLU and experiment with a diverse set of pretrained language representations for Intent Recognition and Entity Extraction tasks in the math learning domain. To recognize kids' speech in realistic home environments, we investigate several ASR systems, including the commercial Google Cloud and the latest open-source Whisper solutions with varying model sizes. We evaluate the SLU pipeline by testing our best-performing NLU models on noisy ASR output to inspect the challenges of understanding children for math learning in authentic homes.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Intent RecognitionMathNatural Language Understandingspeech-recognitionSpeech RecognitionSpoken Language UnderstandingSimilar Papers 제목 키워드 기반
Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System
Contextually aware intelligent agents are often required to understand the users and their surroundings in real-time. Our goal is to build Artificial Intelligence (AI) systems that can assist children in their learning p…
Data AugmentationIntent RecognitionMathNatural Language Understanding+2Weak-Supervised Dysarthria-invariant Features for Spoken Language Understanding using an FHVAE and Adversarial Training
The scarcity of training data and the large speaker variation in dysarthric speech lead to poor accuracy and poor speaker generalization of spoken language understanding systems for dysarthric speech. Through work on the…
Spoken Language Understanding'Beach' to 'Bitch': Inadvertent Unsafe Transcription of Kids' Content on YouTube
Over the last few years, YouTube Kids has emerged as one of the highly competitive alternatives to television for children's entertainment. Consequently, YouTube Kids' content should receive an additional level of scruti…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Hallucinationspeech-recognition+1Joint Learning of Dialog Act Segmentation and Recognition in Spoken Dialog Using Neural Networks
Dialog act segmentation and recognition are basic natural language understanding tasks in spoken dialog systems. This paper investigates a unified architecture for these two tasks, which aims to improve the model{'}s per…
Automatic Speech Recognition (ASR)Natural Language UnderstandingSegmentationSpeech Recognition+1Learn to Understand Negation in Video Retrieval
Negation is a common linguistic skill that allows human to express what we do NOT want. Naturally, one might expect video retrieval to support natural-language queries with negation, e.g., finding shots of kids sitting o…
Natural Language QueriesNegationRetrievalTriplet+2