Instruction Finetuning for Leaderboard Generation from Empirical AI Research
This study demonstrates the application of instruction finetuning of pretrained Large Language Models (LLMs) to automate the generation of AI research leaderboards, extracting (Task, Dataset, Metric, Score) quadruples from articles. It aims to streamline the dissemination of advancements in AI research by transitioning from traditional, manual community curation, or otherwise taxonomy-constrained natural language inference (NLI) models, to an automated, generative LLM-based approach. Utilizing the FLAN-T5 model, this research enhances LLMs' adaptability and reliability in information extraction, offering a novel method for structured knowledge representation.
Code (0)
등록된 구현이 없습니다.
Tasks
ArticlesNatural Language InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Effective Context Selection in LLM-based Leaderboard Generation: An Empirical Study
This paper explores the impact of context selection on the efficiency of Large Language Models (LLMs) in generating Artificial Intelligence (AI) research leaderboards, a task defined as the extraction of (Task, Dataset, …
ArticlesNatural Language InferenceText GenerationGenQA: Generating Millions of Instructions from a Handful of Prompts
Most public instruction finetuning datasets are relatively small compared to the closed source datasets used to train industry models. To study questions about finetuning at scale, such as curricula and learning rate coo…
Instruction Mining: Instruction Data Selection for Tuning Large Language Models
Large language models (LLMs) are initially pretrained for broad capabilities and then finetuned with instruction-following datasets to improve their performance in interacting with humans. Despite advances in finetuning,…
Instruction FollowingLanguage ModelingLanguage ModellingLarge Language ModelParameter Efficient Instruction Tuning: An Empirical Study
Instruction tuning has become an important step for finetuning pretrained language models to better follow human instructions and generalize on various tasks. Nowadays, pretrained language models become increasingly larg…
Instruction FollowingMemorizationSelf-Alignment with Instruction Backtranslation
We present a scalable method to build a high quality instruction following language model by automatically labelling human-written text with corresponding instructions. Our approach, named instruction backtranslation, st…
Instruction FollowingLanguage ModelingLanguage Modelling