RECIPE: Applying Open Domain Question Answering to Privacy Policies
We describe our experiences in using an open domain question answering model (Chen et al., 2017) to evaluate an out-of-domain QA task of assisting in analyzing privacy policies of companies. Specifically, Relevant CI Parameters Extractor (RECIPE) seeks to answer questions posed by the theory of contextual integrity (CI) regarding the information flows described in the privacy statements. These questions have a simple syntactic structure and the answers are factoids or descriptive in nature. The model achieved an F1 score of 72.33, but we noticed that combining the results of this model with a neural dependency parser based approach yields a significantly higher F1 score of 92.35 compared to manual annotations. This indicates that future work which in-corporates signals from parsing like NLP tasks more explicitly can generalize better on out-of-domain tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
DescriptiveOpen-Domain Question AnsweringQuestion AnsweringReading ComprehensionSimilar Papers 제목 키워드 기반
RAFT: Adapting Language Model to Domain Specific RAG
Pretraining Large Language Models (LLMs) on large corpora of textual data is now a standard paradigm. When using these LLMs for many downstream applications, it is common to additionally bake in new knowledge (e.g., time…
Language ModelingLanguage ModellingmodelRAGImproving Retrieval Augmented Open-Domain Question-Answering with Vectorized Contexts
In the era of large language models, applying techniques such as Retrieval Augmented Generation can better address Open-Domain Question-Answering problems. Due to constraints including model sizes and computing resources…
In-Context LearningLanguage ModelingLanguage ModellingOpen-Domain Question Answering+3Adaptive Information Seeking for Open-Domain Question Answering
Information seeking is an essential step for open-domain question answering to efficiently gather evidence from a large corpus. Recently, iterative approaches have been proven to be effective for complex questions, by re…
Open-Domain Question AnsweringQuestion AnsweringRetrievalEDUQA: Educational Domain Question Answering System using Conceptual Network Mapping
Most of the existing question answering models can be largely compiled into two categories: i) open domain question answering models that answer generic questions and use large-scale knowledge base along with the targete…
Answer GenerationOpen-Domain Question AnsweringQuestion AnsweringRetrievalQuestion Answering over Electronic Devices: A New Benchmark Dataset and a Multi-Task Learning based QA Framework
Answering questions asked from instructional corpora such as E-manuals, recipe books, etc., has been far less studied than open-domain factoid context-based question answering. This can be primarily attributed to the abs…
Community Question AnsweringMulti-Task LearningQuestion Answering