Papers Multi-task Language Understanding
“Multi-task Language Understanding” 태그가 달린 논문 57편 · 필터 해제
Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
Large language models (LLMs) have demonstrated considerable proficiency in general natural language processing (NLP) tasks. Instruction tuning, a successful paradigm, enhances the ability of LLMs to follow natural langua…
Arithmetic ReasoningCode GenerationCommon Sense ReasoningGPU+5Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitabl…
1 Image, 2*2 StitchingArithmetic ReasoningCode GenerationImage Retrieval+5The Falcon Series of Open Language Models
We introduce the Falcon series: 7B, 40B, and 180B parameters causal decoder-only models trained on a diverse high-quality corpora predominantly assembled from web data. The largest model, Falcon-180B, has been trained on…
DecoderMulti-task Language UnderstandingSentence CompletionOrca 2: Teaching Small Language Models How to Reason
Orca 1 learns from rich signals, such as explanation traces, allowing it to outperform conventional instruction-tuned models on benchmarks like BigBench Hard and AGIEval. In Orca 2, we continue exploring how improved tra…
Arithmetic ReasoningCommon Sense ReasoningcounterfactualCounterfactual Reasoning+6MiLe Loss: a New Loss for Mitigating the Bias of Learning Difficulties in Generative Language Models
Generative language models are usually pretrained on large text corpus via predicting the next token (i.e., sub-word/word/phrase) given the previous ones. Recent works have demonstrated the impressive performance of larg…
Language ModelingLanguage ModellingMulti-task Language UnderstandingMistral 7B
We introduce Mistral 7B v0.1, a 7-billion-parameter language model engineered for superior performance and efficiency. Mistral 7B outperforms Llama 2 13B across all evaluated benchmarks, and Llama 1 34B in reasoning, mat…
answerability predictionArithmetic ReasoningChatbotCode Generation+11Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU
Although large language models (LLMs) are often pre-trained on large-scale multilingual texts, their reasoning abilities and real-world knowledge are mainly evaluated based on English datasets. Assessing LLM capabilities…
Multi-task Language UnderstandingWorld KnowledgeAre Human-generated Demonstrations Necessary for In-context Learning?
Despite the promising few-shot ability of large language models (LLMs), the standard paradigm of In-context Learning (ICL) suffers the disadvantages of susceptibility to selected demonstrations and the intricacy to gener…
Arithmetic ReasoningCode GenerationDecision MakingIn-Context Learning+2Textbooks Are All You Need II: phi-1.5 technical report
We continue the investigation into the power of smaller Transformer-based language models as initiated by \textbf{TinyStories} -- a 10 million parameter model that can produce coherent English -- and the follow-up work o…
AllCode GenerationCommon Sense ReasoningIn-Context Learning+2Llama 2: Open Foundation and Fine-Tuned Chat Models
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are…
Arithmetic ReasoningCode GenerationMath Word Problem Solving+4Model Card and Evaluations for Claude Models
This report includes the model card [1] for Claude models, focusing on Claude 2, along with the results of a range of safety, alignment, and capabilities evaluations. We have been iterating on the training and evaluation…
Arithmetic ReasoningBug fixingCode GenerationCommon Sense Reasoning+8Let's Do a Thought Experiment: Using Counterfactuals to Improve Moral Reasoning
Language models still struggle on moral reasoning, despite their impressive performance in many other tasks. In particular, the Moral Scenarios task in MMLU (Multi-task Language Understanding) is among the worst performi…
counterfactualMathMMLUMoral Scenarios+1MERGE: Fast Private Text Generation
The drastic increase in language models' parameters has led to a new trend of deploying models in cloud servers, raising growing concerns about private inference for Transformer-based models. Existing two-party privacy-p…
Code CompletionMulti-task Language UnderstandingNatural Language UnderstandingPrivacy Preserving+1PaLM 2 Technical Report
We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is a Transformer-based model trained usin…
Code GenerationCommon Sense ReasoningCoreference ResolutionLanguage Modelling+4BloombergGPT: A Large Language Model for Finance
The use of NLP in the realm of financial technology is broad and complex, with applications ranging from sentiment analysis and named entity recognition to question answering. Large Language Models (LLMs) have been shown…
Causal JudgmentCommon Sense ReasoningDate UnderstandingDisambiguation QA+25GPT-4 Technical Report
We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-world scenarios, GPT-4 exhibits human-level…
answerability predictionArithmetic ReasoningBug fixingCode Generation+18LLaMA: Open and Efficient Foundation Language Models
We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly a…
Arithmetic ReasoningCode GenerationCommon Sense ReasoningFew-Shot Learning+6REPLUG: Retrieval-Augmented Black-Box Language Models
We introduce REPLUG, a retrieval-augmented language modeling framework that treats the language model (LM) as a black box and augments it with a tuneable retrieval model. Unlike prior retrieval-augmented LMs that train l…
Language ModelingLanguage ModellingMMLUMulti-task Language Understanding+2Galactica: A Large Language Model for Science
Information overload is a major obstacle to scientific progress. The explosive growth in scientific literature and data has made it ever harder to discover useful insights in a large mass of information. Today scientific…
AnachronismsBias DetectionCitation PredictionClassification+21Transcending Scaling Laws with 0.1% Extra Compute
Scaling language models improves performance but comes with significant computational costs. This paper proposes UL2R, a method that substantially improves existing language models and their scaling curves with a relativ…
Arithmetic ReasoningCross-Lingual Question AnsweringGSM8KLanguage Modelling+4