Does Instruction Tuning Make LLMs More Consistent?

2024-04-23 · Constanza Fierro, Jiaang Li, Anders Søgaard

The purpose of instruction tuning is enabling zero-shot performance, but instruction tuning has also been shown to improve chain-of-thought reasoning and value alignment (Si et al., 2023). Here we consider the impact on $\textit{consistency}$, i.e., the sensitivity of language models to small perturbations in the input. We compare 10 instruction-tuned LLaMA models to the original LLaMA-7b model and show that almost across-the-board they become more consistent, both in terms of their representations and their predictions in zero-shot and downstream tasks. We explain these improvements through mechanistic analyses of factual recall.

📄 PDF Abstract BibTeX arXiv:2404.15206

Code (0)

등록된 구현이 없습니다.

Tasks

Sensitivity

Methods 이 논문이 사용한 방법론

LLaMA LLaMA is a collection of foundation language models ranging from 7B to 65B parameters. It is based on the transformer architecture with various improvements that were…

Psychometric Predictive Power of Large Language Models

2023-11-13 · Tatsuki Kuribayashi, Yohei Oseki, Timothy Baldwin

Instruction tuning aligns the response of large language models (LLMs) with human preferences. Despite such efforts in human--LLM alignment, we find that instruction tuning does not always make LLMs human-like from a cog…

Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions

2023-09-14 · Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röttger 외

Training large language models to follow instructions makes them perform better on a wide range of tasks and generally become more helpful. However, a perfectly helpful model will follow even the most malicious instructi…

Fine-Tuning on Noisy Instructions: Effects on Generalization and Performance

2025-10-03 · Ahmed Alajrami, Xingwei Tan, Nikolaos Aletras arxiv

Instruction-tuning plays a vital role in enhancing the task-solving abilities of large language models (LLMs), improving their usability in generating helpful responses on various tasks. However, previous work has demons…

Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions?

2024-02-21 · Alexander Arno Weber, Klaudia Thellmann, Jan Ebert, Nicolas Flores-Herr 외

The adaption of multilingual pre-trained LLMs into eloquent and helpful assistants is essential to facilitate their use across different language regions. In that spirit, we are the first to conduct an extensive study of…

Instruction Following

An Empirical Study of Instruction-tuning Large Language Models in Chinese

2023-10-11 · Qingyi Si, Tong Wang, Zheng Lin, Xu Zhang 외

The success of ChatGPT validates the potential of large language models (LLMs) in artificial general intelligence (AGI). Subsequently, the release of LLMs has sparked the open-source community's interest in instruction-t…