Mini-Giants: "Small" Language Models and Open Source Win-Win
ChatGPT is phenomenal. However, it is prohibitively expensive to train and refine such giant models. Fortunately, small language models are flourishing and becoming more and more competent. We call them "mini-giants". We argue that open source community like Kaggle and mini-giants will win-win in many ways, technically, ethically and socially. In this article, we present a brief yet rich background, discuss how to attain small language models, present a comparative study of small language models and a brief discussion of evaluation methods, discuss the application scenarios where small language models are most needed in the real world, and conclude with discussion and outlook.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
GIANTS: Generative Insight Anticipation from Scientific Literature
Scientific breakthroughs often emerge from synthesizing prior ideas into novel contributions. While language models (LMs) show promise in scientific discovery, their ability to perform this targeted, literature-grounded …
Reinforcement LearningThe Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giants
As proprietary giants increasingly dominate the race for ever-larger language models, a pressing question arises for the open-source community: can smaller models remain competitive across a broad range of tasks? In this…
ClusteringOut-of-Distribution GeneralizationSemantic SimilaritySemantic Textual SimilarityLittle Giants: Exploring the Potential of Small LLMs as Evaluation Metrics in Summarization in the Eval4NLP 2023 Shared Task
This paper describes and analyzes our participation in the 2023 Eval4NLP shared task, which focuses on assessing the effectiveness of prompt-based techniques to empower Large Language Models to handle the task of quality…
One-Shot LearningWEST: LLM based Speech Toolkit for Speech Understanding, Generation, and Interaction
In this paper, we present WEST(WE Speech Toolkit), a speech toolkit based on a large language model (LLM) for speech understanding, generation, and interaction. There are three key features of WEST: 1) Fully LLM-based: S…
Breaking the Myth: Can Small Models Infer Postconditions Too?
Formal specifications are essential for ensuring software correctness, yet manually writing them is tedious and error-prone. Large Language Models (LLMs) have shown promise in generating such specifications from natural …