Probabilistic Reasoning with LLMs for k-anonymity Estimation
Probabilistic reasoning is a key aspect of both human and artificial intelligence that allows for handling uncertainty and ambiguity in decision-making. In this paper, we introduce a new numerical reasoning task under uncertainty for large language models, focusing on estimating the privacy risk of user-generated documents containing privacy-sensitive information. We propose BRANCH, a new LLM methodology that estimates the k-privacy value of a text-the size of the population matching the given information. BRANCH factorizes a joint probability distribution of personal information as random variables. The probability of each factor in a population is estimated separately using a Bayesian network and combined to compute the final k-value. Our experiments show that this method successfully estimates the k-value 73% of the time, a 13% increase compared to o3-mini with chain-of-thought reasoning. We also find that LLM uncertainty is a good indicator for accuracy, as high-variance predictions are 37.47% less accurate on average.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingPrediction IntervalsSimilar Papers 제목 키워드 기반
Reasoning Under Uncertainty: Exploring Probabilistic Reasoning Capabilities of LLMs
Despite widespread success in language understanding and generation, large language models (LLMs) exhibit unclear and often inconsistent behavior when faced with tasks that require probabilistic reasoning. In this work, …
Using Probabilistic Programs to Train Inductive Reasoning in Large Language Models
Post-training Large Language Models (LLMs) for reasoning typically focuses on deductive tasks such as mathematics and coding where correctness is verifiable. Yet, many real-world reasoning problems are inductive: agents …
OrLog: Resolving Complex Queries with LLMs and Probabilistic Reasoning
Resolving complex information needs that come with multiple constraints should consider enforcing the logical operators encoded in the query (i.e., conjunction, disjunction, negation) on the candidate answer set. Current…
Information RetrievalLogical ReasoningFormal LogicDensity estimation with LLMs: a geometric investigation of in-context learning trajectories
Large language models (LLMs) demonstrate remarkable emergent abilities to perform in-context learning across various tasks, including time series forecasting. This work investigates LLMs' ability to estimate probability …
Density EstimationIn-Context LearningTime Series ForecastingReasoning over Uncertain Text by Generative Large Language Models
This paper considers the challenges Large Language Models (LLMs) face when reasoning over text that includes information involving uncertainty explicitly quantified via probability values. This type of reasoning is relev…
Decision MakingMathematical ReasoningQuestion Answering