Evaluating Information Retrieval Systems for Kids
Evaluation of information retrieval systems (IRS) is a prominent topic among information retrieval researchers--mainly directed at a general population. Children require unique IRS and by extension different ways to evaluate these systems, but as a large population that use IRS have largely been ignored on the evaluation front. In this position paper, we explore many perspectives that must be considered when evaluating IRS; we specially discuss problems faced by researchers who work with children IRS, including lack of evaluation frameworks, limitations of data, and lack of user judgment understanding.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalPositionRetrievalSimilar Papers 제목 키워드 기반
'Beach' to 'Bitch': Inadvertent Unsafe Transcription of Kids' Content on YouTube
Over the last few years, YouTube Kids has emerged as one of the highly competitive alternatives to television for children's entertainment. Consequently, YouTube Kids' content should receive an additional level of scruti…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Hallucinationspeech-recognition+1New Kids on the Block: On the impact of information retrieval on contextual resource integration patterns
The rise of new modes of interaction with AI skyrocketed the popularity, applicability, and amount of use cases. Despite this evolution, conceptual integration is falling behind. Studies suggest that there is hardly a sy…
Information RetrievalRetrievalK-Function: Joint Pronunciation Transcription and Feedback for Evaluating Kids Language Function
Evaluating young children's language is challenging for automatic speech recognizers due to high-pitched voices, prolonged sounds, and limited data. We introduce K-Function, a framework that combines accurate sub-word tr…
Learn to Understand Negation in Video Retrieval
Negation is a common linguistic skill that allows human to express what we do NOT want. Naturally, one might expect video retrieval to support natural-language queries with negation, e.g., finding shots of kids sitting o…
Natural Language QueriesNegationRetrievalTriplet+2Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home
Enriching the quality of early childhood education with interactive math learning at home systems, empowered by recent advances in conversational AI technologies, is slowly becoming a reality. With this motivation, we im…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Intent RecognitionMath+4