Linguistic Properties of Truthful Response
We investigate the phenomenon of an LLM's untruthful response using a large set of 220 handcrafted linguistic features. We focus on GPT-3 models and find that the linguistic profiles of responses are similar across model sizes. That is, how varying-sized LLMs respond to given prompts stays similar on the linguistic properties level. We expand upon this finding by training support vector machines that rely only upon the stylistic components of model responses to classify the truthfulness of statements. Though the dataset size limits our current findings, we show the possibility that truthfulness detection is possible without evaluating the content itself. But at the same time, the limited scope of our experiments must be taken into account in interpreting the results.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Linguistic Cues to Deception and Perceived Deception in Interview Dialogues
We explore deception detection in interview dialogues. We analyze a set of linguistic features in both truthful and deceptive responses to interview questions. We also study the perception of deception, identifying chara…
BIG-bench Machine LearningDeception DetectionGeneral ClassificationWhen lies are mostly truthful: automated verbal deception detection for embedded lies
Background: Verbal deception detection research relies on narratives and commonly assumes statements as truthful or deceptive. A more realistic perspective acknowledges that the veracity of statements exists on a continu…
Deception DetectionThe Square Root Agreement Rule for Incentivizing Truthful Feedback on Online Platforms
A major challenge in obtaining evaluations of products or services on e-commerce platforms is eliciting informative responses in the absence of verifiability. This paper proposes the Square Root Agreement Rule (SRA): a s…
TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space
Large Language Models (LLMs) sometimes suffer from producing hallucinations, especially LLMs may generate untruthful responses despite knowing the correct knowledge. Activating the truthfulness within LLM is the key to f…
Contrastive LearningHallucinationHallucination EvaluationLanguage Modelling+4Acoustic-Prosodic and Lexical Cues to Deception and Trust: Deciphering How People Detect Lies
Humans rarely perform better than chance at lie detection. To better understand human perception of deception, we created a game framework, LieCatcher, to collect ratings of perceived deception using a large corpus of de…
Deception Detection