Question Rephrasing for Quantifying Uncertainty in Large Language Models: Applications in Molecular Chemistry Tasks
Uncertainty quantification enables users to assess the reliability of responses generated by large language models (LLMs). We present a novel Question Rephrasing technique to evaluate the input uncertainty of LLMs, which refers to the uncertainty arising from equivalent variations of the inputs provided to LLMs. This technique is integrated with sampling methods that measure the output uncertainty of LLMs, thereby offering a more comprehensive uncertainty assessment. We validated our approach on property prediction and reaction prediction for molecular chemistry tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionProperty PredictionUncertainty QuantificationSimilar Papers 제목 키워드 기반
Quantifying the Risks of Tool-assisted Rephrasing to Linguistic Diversity
Writing assistants and large language models see widespread use in the creation of text content. While their effectiveness for individual users has been evaluated in the literature, little is known about their proclivity…
DiversityJust rephrase it! Uncertainty estimation in closed-source language models via multiple rephrased queries
State-of-the-art large language models are sometimes distributed as open-source software but are also increasingly provided as a closed-source service. These closed-source large-language models typically see the widest u…
Can Pre-training help VQA with Lexical Variations?
Rephrasings or paraphrases are sentences with similar meanings expressed in different ways. Visual Question Answering (VQA) models are closing the gap with the oracle performance for datasets like VQA2.0. However, these …
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Transformation-Augmented GRPO for Enhancing Exploration in Reasoning of Large Language Models
Group Relative Policy Optimization (GRPO) has become the dominant method for reinforcement learning with verifiable rewards in large language models, but it suffers from two critical limitations: gradient vanishing and d…
Reinforcement LearningWhy Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA
Natural language is ambiguous. Resolving ambiguous questions is key to successfully answering them. Focusing on questions about images, we create a dataset of ambiguous examples. We annotate these, grouping answers by th…
Question GenerationQuestion-GenerationVisual Question Answering (VQA)