Product Answer Generation from Heterogeneous Sources: A New Benchmark and Best Practices
It is of great value to answer product questions based on heterogeneous information sources available on web product pages, e.g., semi-structured attributes, text descriptions, user-provided contents, etc. However, these sources have different structures and writing styles, which poses challenges for (1) evidence ranking, (2) source selection, and (3) answer generation. In this paper, we build a benchmark with annotations for both evidence selection and answer generation covering 6 information sources. Based on this benchmark, we conduct a comprehensive study and present a set of best practices. We show that all sources are important and contribute to answering questions. Handling all sources within one single model can produce comparable confidence scores across sources and combining multiple sources for training always helps, even for sources with totally different structures. We further propose a novel data augmentation method to iteratively create training samples for answer generation, which achieves close-to-human performance with only a few thousandannotations. Finally, we perform an in-depth error analysis of model predictions and highlight the challenges for future research.
Code (0)
등록된 구현이 없습니다.
Tasks
Answer GenerationData AugmentationEvidence SelectionSimilar Papers 제목 키워드 기반
Product Answer Generation from Heterogeneous Sources: A New Benchmark and Best Practices
It is of great value to answer product questions based on heterogeneous information sources available on web product pages, e.g., semi-structured attributes, text descriptions, user-provided contents, etc. However, these…
Answer GenerationData AugmentationEvidence SelectionAnswering Product-related Questions with Heterogeneous Information
Providing instant response for product-related questions in E-commerce question answering platforms can greatly improve users{'} online shopping experience. However, existing product question answering (PQA) methods only…
AttributeQuestion AnsweringContextually Aware E-Commerce Product Question Answering using RAG
E-commerce product pages contain a mix of structured specifications, unstructured reviews, and contextual elements like personalized offers or regional variants. Although informative, this volume can lead to cognitive ov…
Question AnsweringCompMix: A Benchmark for Heterogeneous Question Answering
Fact-centric question answering (QA) often requires access to multiple, heterogeneous, information sources. By jointly considering several sources like a knowledge base (KB), a text collection, and tables from the web, Q…
Question AnsweringNatural Answer Generation with Heterogeneous Memory
Memory augmented encoder-decoder framework has achieved promising progress for natural language generation tasks. Such frameworks enable a decoder to retrieve from a memory during generation. However, less research has b…
Answer GenerationDecoderQuestion AnsweringSentence+1