Papers Small Language Model
“Small Language Model” 태그가 달린 논문 109편 · 필터 해제
Personal Intelligence System UniLM: Hybrid On-Device Small Language Model and Server-Based Large Language Model for Malay Nusantara
In contexts with limited computational and data resources, high-resource language models often prove inadequate, particularly when addressing the specific needs of Malay languages. This paper introduces a Personal Intell…
Language ModelingLanguage ModellingLarge Language ModelMachine Translation+3Generating long-horizon stock "buy" signals with a neural language model
This paper describes experiments on fine-tuning a small language model to generate forecasts of long-horizon stock price movements. Inputs to the model are narrative text from 10-K reports of large market capitalization …
Language ModelingLanguage ModellingSmall Language ModelDecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models
The performance of Large Language Models (LLMs) is substantially influenced by the pretraining corpus, which consists of vast quantities of unsupervised data processed by the models. Despite its critical role in model pe…
Language ModelingLanguage ModellingLarge Language ModelSmall Language ModelCross-lingual Transfer for Automatic Question Generation by Learning Interrogative Structures in Target Languages
Automatic question generation (QG) serves a wide range of purposes, such as augmenting question-answering (QA) corpora, enhancing chatbot systems, and developing educational materials. Despite its importance, most existi…
ChatbotCross-Lingual TransferLanguage ModelingLanguage Modelling+4Small Language Models are Equation Reasoners
Chain-of-Thought (CoT) reasoning has enabled Large Language Model (LLM) to achieve remarkable performance in various NLP tasks, including arithmetic problem-solving. However, this success does not generalize to small lan…
Arithmetic ReasoningKnowledge DistillationLanguage ModelingLanguage Modelling+2Cross-Domain Content Generation with Domain-Specific Small Language Models
Generating domain-specific content using small language models poses challenges, especially when dealing with multiple distinct datasets with minimal overlap. In this study, we explore methods to enable a small language …
Small Language ModelSmall Language Models can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs
In this paper, we evaluate the creative fiction writing abilities of a fine-tuned small language model (SLM), BART-large, and compare its performance to human writers and two large language models (LLMs): GPT-3.5 and GPT…
Language ModellingSmall Language ModelAnyMatch -- Efficient Zero-Shot Entity Matching with a Small Language Model
Entity matching (EM) is the problem of determining whether two records refer to same real-world entity, which is crucial in data integration, e.g., for product catalogs or address databases. A major drawback of many EM a…
AttributeAutoMLData IntegrationLanguage Modeling+3TinyAgent: Function Calling at the Edge
Recent large language models (LLMs) have enabled the development of advanced agentic systems that can integrate various tools and APIs to fulfill user queries through function calling. However, the deployment of these LL…
Language ModellingQuantizationSmall Language ModelAssessing Generative Language Models in Classification Tasks: Performance and Self-Evaluation Capabilities in the Environmental and Climate Change Domain
This paper examines the performance of two Large Language Models (LLMs), GPT3.5 and Llama2 and one Small Language Model (SLM) Gemma, across three different classification tasks within the climate change (CC) and environm…
Language ModelingLanguage ModellingSmall Language Modeltext-classification+1InkubaLM: A small language model for low-resource African languages
High-resource language models often fall short in the African context, where there is a critical need for models that are efficient, accessible, and locally relevant, even amidst significant computing and data constraint…
Language ModelingLanguage ModellingMachine TranslationQuestion Answering+2LlamaDuo: LLMOps Pipeline for Seamless Migration from Service LLMs to Small-Scale Local LLMs
The widespread adoption of cloud-based proprietary large language models (LLMs) has introduced significant challenges, including operational dependencies, privacy concerns, and the necessity of continuous internet connec…
Language ModelingLanguage ModellingSmall Language ModelSLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detection
Large language models (LLMs) are highly capable but face latency challenges in real-time applications, such as conducting online hallucination detection. To overcome this issue, we propose a novel framework that leverage…
HallucinationLanguage ModelingLanguage ModellingSmall Language ModelLeveraging Fine-Tuned Retrieval-Augmented Generation with Long-Context Support: For 3GPP Standards
Recent studies show that large language models (LLMs) struggle with technical standards in telecommunications. We propose a fine-tuned retrieval-augmented generation (RAG) system based on the Phi-2 small language model (…
ChunkingComputational EfficiencyLanguage ModellingQuestion Answering+5VersusDebias: Universal Zero-Shot Debiasing for Text-to-Image Models via SLM-Based Prompt Engineering and Generative Adversary
With the rapid development of Text-to-Image (T2I) models, biases in human image generation against demographic social groups become a significant concern, impacting fairness and ethical standards in AI. Some researchers …
AttributeFairnessImage GenerationLanguage Modelling+2Exploring Domain Robust Lightweight Reward Models based on Router Mechanism
Recent advancements in large language models have heavily relied on the large reward model from reinforcement learning from human feedback for fine-tuning. However, the use of a single reward model across various domains…
Language ModelingLanguage ModellingMixture-of-ExpertsSmall Language ModelGraph-Structured Speculative Decoding
Speculative decoding has emerged as a promising technique to accelerate the inference of Large Language Models (LLMs) by employing a small language model to draft a hypothesis sequence, which is then validated by the LLM…
Language ModellingSmall Language ModelRDBE: Reasoning Distillation-Based Evaluation Enhances Automatic Essay Scoring
Recently, various encoder-only and encoder-decoder pre-trained models like BERT and T5 have been applied to automatic essay scoring (AES) as small language models. However, existing studies have primarily treated this ta…
DecoderLanguage ModelingLanguage ModellingLarge Language Model+1TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddings
The goal of text style transfer is to transform the style of texts while preserving their original meaning, often with only a few examples of the target style. Existing style transfer methods generally rely on the few-sh…
AttributeLanguage ModelingLanguage ModellingSmall Language Model+3PDSS: A Privacy-Preserving Framework for Step-by-Step Distillation of Large Language Models
In the context of real-world applications, leveraging large language models (LLMs) for domain-specific tasks often faces two major challenges: domain-specific knowledge privacy and constrained resources. To address these…
DecoderLanguage ModelingLanguage ModellingMulti-Task Learning+3