Trusted Source Alignment in Large Language Models
Large language models (LLMs) are trained on web-scale corpora that inevitably include contradictory factual information from sources of varying reliability. In this paper, we propose measuring an LLM property called trusted source alignment (TSA): the model's propensity to align with content produced by trusted publishers in the face of uncertainty or controversy. We present FactCheckQA, a TSA evaluation dataset based on a corpus of fact checking articles. We describe a simple protocol for evaluating TSA and offer a detailed analysis of design considerations including response extraction, claim contextualization, and bias in prompt formulation. Applying the protocol to PaLM-2, we find that as we scale up the model size, the model performance on FactCheckQA improves from near-random to up to 80% balanced accuracy in aligning with trusted sources.
Code (0)
등록된 구현이 없습니다.
Tasks
ArticlesFact CheckingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats
As large language models (LLMs) become increasingly capable, it is prudent to assess whether safety measures remain effective even if LLMs intentionally try to bypass them. Previous work introduced control evaluations, a…
Code GenerationTraceGuard: Structured Multi-Dimensional Monitoring as a Collusion-Resistant Control Protocol
AI control protocols use monitors to detect attacks by untrusted AI agents, but standard single-score monitors face two limitations: they miss subtle attacks where outputs look clean but reasoning is off, and they collap…
Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
The growing importance of massive datasets used for deep learning makes robustness to label noise a critical property for classifiers to have. Sources of label noise include automatic labeling, non-expert labeling, and l…
Data PoisoningDetecting fake news for the new coronavirus by reasoning on the Covid-19 ontology
In the context of the Covid-19 pandemic, many were quick to spread deceptive information. I investigate here how reasoning in Description Logics (DLs) can detect inconsistencies between trusted medical sources and not tr…
Cross-Region Domain Adaptation for Class-level Alignment
Semantic segmentation requires a lot of training data, which necessitates costly annotation. There have been many studies on unsupervised domain adaptation (UDA) from one domain to another, e.g., from computer graphics t…
Domain AdaptationImage-to-Image TranslationSemantic SegmentationSynthetic-to-Real Translation+1