A Theoretical Understanding of Self-Correction through In-context Alignment

2024-05-28 · Yifei Wang, Yuyang Wu, Zeming Wei, Stefanie Jegelka, Yisen Wang

Going beyond mimicking limited human experiences, recent studies show initial evidence that, like humans, large language models (LLMs) are capable of improving their abilities purely by self-correction, i.e., correcting previous responses through self-examination, in certain circumstances. Nevertheless, little is known about how such capabilities arise. In this work, based on a simplified setup akin to an alignment task, we theoretically analyze self-correction from an in-context learning perspective, showing that when LLMs give relatively accurate self-examinations as rewards, they are capable of refining responses in an in-context way. Notably, going beyond previous theories on over-simplified linear transformers, our theoretical construction underpins the roles of several key designs of realistic transformers for self-correction: softmax attention, multi-head attention, and the MLP block. We validate these findings extensively on synthetic datasets. Inspired by these findings, we also illustrate novel applications of self-correction, such as defending against LLM jailbreaks, where a simple self-correction step does make a large difference. We believe that these findings will inspire further research on understanding, exploiting, and enhancing self-correction for building better foundation models.

📄 PDF Abstract BibTeX arXiv:2405.18634

Code (0)

등록된 구현이 없습니다.

Tasks

In-Context Learning

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

A Probabilistic Inference Scaling Theory for LLM Self-Correction

2025-08-22 · Zhe Yang, Yichang Zhang, Yudong Wang, Ziyao Xu 외 arxiv

Large Language Models (LLMs) have demonstrated the capability to refine their generated answers through self-correction, enabling continuous performance improvement over multiple rounds. However, the mechanisms underlyin…

Smaller Large Language Models Can Do Moral Self-Correction

2024-10-30 · Guangliang Liu, Zhiyu Xue, Rongrong Wang, Kristen Marie Johnson

Self-correction is one of the most amazing emerging capabilities of Large Language Models (LLMs), enabling LLMs to self-modify an inappropriate output given a natural language feedback which describes the problems of tha…

Language Modeling Language Modelling Safety Alignment

Corrective In-Context Learning: Evaluating Self-Correction in Large Language Models

2025-03-20 · Mario Sanz-Guerrero, Katharina von der Wense

In-context learning (ICL) has transformed the use of large language models (LLMs) for NLP tasks, enabling few-shot learning by conditioning on labeled examples without finetuning. Despite its effectiveness, ICL is prone …

Few-Shot Learning In-Context Learning text-classification Text Classification

Discourse Heuristics For Paradoxically Moral Self-Correction

2025-07-01 · Guangliang Liu, Zimo Qi, Xitong Zhang, Kristen Marie Johnson arxiv

Moral self-correction has emerged as a promising approach for aligning the output of Large Language Models (LLMs) with human moral values. However, moral self-correction techniques are subject to two primary paradoxes. F…

Confidence v.s. Critique: A Decomposition of Self-Correction Capability for LLMs

2024-12-27 · Zhe Yang, Yichang Zhang, Yudong Wang, Ziyao Xu 외

Large Language Models (LLMs) can correct their self-generated responses, but a decline in accuracy after self-correction is also witnessed. To have a deeper understanding of self-correction, we endeavor to decompose, eva…