Papers Small Language Model
“Small Language Model” 태그가 달린 논문 109편 · 필터 해제
HARE: HumAn pRiors, a key to small language model Efficiency
Human priors play a crucial role in efficiently utilizing data in deep learning. However, with the development of large language models (LLMs), there is an increasing emphasis on scaling both model size and data volume, …
DiversityLanguage ModelingLanguage ModellingSmall Language ModelSmall-E: Small Language Model with Linear Attention for Efficient Speech Synthesis
Recent advancements in text-to-speech (TTS) powered by language models have showcased remarkable capabilities in achieving naturalness and zero-shot voice cloning. Notably, the decoder-only transformer is the prominent a…
DecoderInductive BiasLanguage ModelingLanguage Modelling+5Efficient Medical Question Answering with Knowledge-Augmented Question Generation
In the expanding field of language model applications, medical knowledge representation remains a significant challenge due to the specialized nature of the domain. Large language models, such as GPT-4, obtain reasonable…
Language ModelingLanguage ModellingMedical Question AnsweringQuestion Answering+3XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples
Recent studies indicate that leveraging off-the-shelf or fine-tuned retrievers, capable of retrieving relevant in-context examples tailored to the input query, enhances few-shot in-context learning of English. However, a…
In-Context LearningLanguage ModelingLanguage ModellingLarge Language Model+5CofiPara: A Coarse-to-fine Paradigm for Multimodal Sarcasm Target Identification with Large Multimodal Models
Social media abounds with multimodal sarcasm, and identifying sarcasm targets is particularly challenging due to the implicit incongruity not directly evident in the text and image modalities. Current methods for Multimo…
Language ModelingLanguage ModellingMultimodal ReasoningSarcasm Detection+1Plug and Play with Prompts: A Prompt Tuning Approach for Controlling Text Generation
Transformer-based Large Language Models (LLMs) have shown exceptional language generation capabilities in response to text-based prompts. However, controlling the direction of generation via textual prompts has been chal…
Language ModelingLanguage ModellingSentiment AnalysisSmall Language Model+1Towards Pareto Optimal Throughput in Small Language Model Serving
Large language models (LLMs) have revolutionized the state-of-the-art of many different natural language processing tasks. Although serving LLMs is computationally and memory demanding, the rise of Small Language Models …
Language ModelingLanguage ModellingSmall Language ModelThink Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding
Large language models (LLMs) have become ubiquitous in practice and are widely used for generation tasks such as translation, summarization and instruction following. However, their enormous size and reliance on autoregr…
DecoderInstruction FollowingLanguage ModelingLanguage Modelling+2Cognitive Visual-Language Mapper: Advancing Multimodal Comprehension with Enhanced Visual Knowledge Alignment
Evaluating and Rethinking the current landscape of Large Multimodal Models (LMMs), we observe that widely-used visual-language projection approaches (e.g., Q-former or MLP) focus on the alignment of image-text descriptio…
Language ModellingQuestion AnsweringSmall Language ModelVisual Question Answering+1Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacement
Speculative decoding is an inference-acceleration method for large language models (LLMs) where a small language model generates a draft-token sequence which is further verified by the target LLM in parallel. Recent work…
Language ModellingSmall Language ModelPurifying Large Language Models by Ensembling a Small Language Model
The emerging success of large language models (LLMs) heavily relies on collecting abundant training data from external (untrusted) sources. Despite substantial efforts devoted to data cleaning and curation, well-construc…
Data PoisoningLanguage ModelingLanguage ModellingSmall Language ModelGrasping the Essentials: Tailoring Large Language Models for Zero-Shot Relation Extraction
Relation extraction (RE) aims to identify semantic relationships between entities within text. Despite considerable advancements, existing models predominantly require extensive annotated training data, which is both cos…
Few-Shot LearningLanguage ModellingRelationRelation Extraction+2Towards Explainable Harmful Meme Detection through Multimodal Debate between Large Language Models
The age of social media is flooded with Internet memes, necessitating a clear grasp and effective identification of harmful ones. This task presents a significant challenge due to the implicit meaning embedded in memes, …
Hateful Meme ClassificationLanguage ModellingSmall Language ModelText GenerationSmall Language Model Meets with Reinforced Vision Vocabulary
Playing Large Vision Language Models (LVLMs) in 2023 is trendy among the AI community. However, the relatively large number of parameters (more than 7B) of popular LVLMs makes it difficult to train and deploy on consumer…
Language ModelingLanguage ModellingLarge Language Modelobject-detection+3Small Language Model Can Self-correct
Generative Language Models (LMs) such as ChatGPT have exhibited remarkable performance across various downstream tasks. Nevertheless, one of their most prominent drawbacks is generating inaccurate or false information wi…
Language ModelingLanguage ModellingmodelSmall Language ModelTinyLlama: An Open-Source Small Language Model
We present TinyLlama, a compact 1.1B language model pretrained on around 1 trillion tokens for approximately 3 epochs. Building on the architecture and tokenizer of Llama 2, TinyLlama leverages various advances contribut…
Computational EfficiencyLanguage ModelingLanguage Modellingmodel+1LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model
In this paper, we introduce LLaVA-$\phi$ (LLaVA-Phi), an efficient multi-modal assistant that harnesses the power of the recently advanced small language model, Phi-2, to facilitate multi-modal dialogues. LLaVA-Phi marks…
Language ModelingLanguage ModellingSmall Language ModelVisual Question AnsweringPerfRL: A Small Language Model Framework for Efficient Code Optimization
Code optimization is a challenging task requiring a substantial level of expertise from developers. Nonetheless, this level of human capacity is not sufficient considering the rapid evolution of new hardware architecture…
Language ModelingLanguage ModellingLarge Language Modelreinforcement-learning+3Recommendations by Concise User Profiles from Review Text
Recommender systems perform well for popular items and users with ample interactions (likes, ratings etc.). This work addresses the difficult and underexplored case of users who have very sparse interactions but post inf…
Language ModelingLanguage ModellingRecommendation SystemsRepresentation Learning+1SOUL: Towards Sentiment and Opinion Understanding of Language
Sentiment analysis is a well-established natural language processing task, with sentiment polarity classification being one of its most popular and representative tasks. However, despite the success of pre-trained langua…
Language ModellingSentiment AnalysisSmall Language Model