xPQA: Cross-Lingual Product Question Answering across 12 Languages
Product Question Answering (PQA) systems are key in e-commerce applications to provide responses to customers' questions as they shop for products. While existing work on PQA focuses mainly on English, in practice there is need to support multiple customer languages while leveraging product information available in English. To study this practical industrial task, we present xPQA, a large-scale annotated cross-lingual PQA dataset in 12 languages across 9 branches, and report results in (1) candidate ranking, to select the best English candidate containing the information to answer a non-English question; and (2) answer generation, to generate a natural-sounding non-English answer based on the selected English candidate. We evaluate various approaches involving machine translation at runtime or offline, leveraging multilingual pre-trained LMs, and including or excluding xPQA training data. We find that (1) In-domain data is essential as cross-lingual rankers trained on other domains perform poorly on the PQA task; (2) Candidate ranking often prefers runtime-translation approaches while answer generation prefers multilingual approaches; (3) Translating offline to augment multilingual models helps candidate ranking mainly on languages with non-Latin scripts; and helps answer generation mainly on languages with Latin scripts. Still, there remains a significant performance gap between the English and the cross-lingual test sets.
Code (1)
Tasks
Answer GenerationMachine TranslationQuestion AnsweringTask 2TranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MedExpQA: Multilingual Benchmarking of Large Language Models for Medical Question Answering
Large Language Models (LLMs) have the potential of facilitating the development of Artificial Intelligence technology to assist medical experts for interactive decision support, which has been demonstrated by their compe…
BenchmarkingMedical Question AnsweringQuestion AnsweringRAG+1Unlocking Markets: A Multilingual Benchmark to Cross-Market Question Answering
Users post numerous product-related questions on e-commerce platforms, affecting their purchase decisions. Product-related question answering (PQA) entails utilizing product-related resources to provide precise responses…
Answer GenerationQuestion AnsweringXOR QA: Cross-lingual Open-Retrieval Question Answering
Multilingual question answering tasks typically assume answers exist in the same language as the question. Yet in practice, many languages face both information scarcity -- where languages have few reference articles -- …
ArticlesMachine TranslationQuestion AnsweringRetrieval+1Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains
Open-retrieval question answering systems are generally trained and tested on large datasets in well-established domains. However, low-resource settings such as new and emerging domains would especially benefit from reli…
ArticlesQuestion AnsweringRetrievalTranslationCross-Lingual Open-Domain Question Answering with Answer Sentence Generation
Open-Domain Generative Question Answering has achieved impressive performance in English by combining document-level retrieval with answer generation. These approaches, which we refer to as GenQA, can generate complete s…
Answer GenerationGenerative Question AnsweringOpen-Domain Question AnsweringQuestion Answering+2