Give me a hint: Can LLMs take a hint to solve math problems?
While state-of-the-art LLMs have shown poor logical and basic mathematical reasoning, recent works try to improve their problem-solving abilities using prompting techniques. We propose giving "hints" to improve the language model's performance on advanced mathematical problems, taking inspiration from how humans approach math pedagogically. We also test robustness to adversarial hints and demonstrate their sensitivity to them. We demonstrate the effectiveness of our approach by evaluating various diverse LLMs, presenting them with a broad set of problems of different difficulties and topics from the MATH dataset and comparing against techniques such as one-shot, few-shot, and chain of thought prompting.
Code (1)
Tasks
Adversarial RobustnessMathMathematical ReasoningSensitivityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Designing and Evaluating Chain-of-Hints for Scientific Question Answering
LLMs are reshaping education, with students increasingly relying on them for learning. Implemented using general-purpose models, these systems are likely to give away the answers, potentially undermining conceptual under…
Question AnsweringWikiHint: A Human-Annotated Dataset for Hint Ranking and Generation
The use of Large Language Models (LLMs) has increased significantly with users frequently asking questions to chatbots. In the time when information is readily accessible, it is crucial to stimulate and preserve human co…
DecoderHint GenerationCan Language Models Take A Hint? Prompting for Controllable Contextualized Commonsense Inference
Generating commonsense assertions within a given story context remains a difficult task for modern language models. Previous research has addressed this problem by aligning commonsense inferences with stories and trainin…
Data AugmentationText GenerationUnspoken Hints: Accuracy Without Acknowledgement in LLM Reasoning
Large language models (LLMs) increasingly rely on chain-of-thought (CoT) prompting to solve mathematical and logical reasoning tasks. Yet, a central question remains: to what extent are these generated rationales \emph{f…
Logical ReasoningStepHint: Multi-level Stepwise Hints Enhance Reinforcement Learning to Reason
Reinforcement learning with verifiable rewards (RLVR) is a promising approach for improving the complex reasoning abilities of large language models (LLMs). However, current RLVR methods face two significant challenges: …
Reinforcement Learning