Papers CoLA
“CoLA” 태그가 달린 논문 78편 · 필터 해제
Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMs
Can a pretrained neural network adapt its architecture to different inputs without any finetuning? Do we need all layers for simple tasks, and are they adequate for challenging tasks? We found that the layers of a pretra…
CoLALarge Language ModelMathLoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing
Recent efforts to combine low-rank adaptation (LoRA) with mixture-of-experts (MoE) for adapting large language models (LLMs) to multiple tasks still exhibit prevailing limitations: they either swap entire attention/feed-…
ARCCoLAGSM8KHumanEval+4CoLA: Collaborative Low-Rank Adaptation
The scaling law of Large Language Models (LLMs) reveals a power-law relationship, showing diminishing return on performance as model scale increases. While training LLMs from scratch is resource-intensive, fine-tuning a …
CoLAMixture-of-Expertsparameter-efficient fine-tuningEnhancing LLM Robustness to Perturbed Instructions: An Empirical Study
Large Language Models (LLMs) are highly vulnerable to input perturbations, as even a small prompt change may result in a substantially different output. Existing methods to enhance LLM robustness are primarily focused on…
CoLADenoisingQNLISST-2CoLa -- Learning to Interactively Collaborate with Large LMs
LLMs' remarkable ability to tackle a wide range of language tasks opened new opportunities for collaborative human-AI problem solving. LLMs can amplify human capabilities by applying their intuitions and reasoning strate…
CoLAText GenerationCatastrophic Forgetting in LLMs: A Comparative Analysis Across Language Tasks
Large Language Models (LLMs) have significantly advanced Natural Language Processing (NLP), particularly in Natural Language Understanding (NLU) tasks. As we progress toward an agentic world where LLM-based agents autono…
CoLAContinual LearningMRPCNatural Language Understanding+2Controlling Large Language Model with Latent Actions
Adapting Large Language Models (LLMs) to downstream tasks using Reinforcement Learning (RL) has proven to be an effective approach. However, LLMs do not inherently define the structure of an agent for RL training, partic…
CoLALanguage ModelingLanguage ModellingLarge Language Model+3CoCo-CoLa: Evaluating and Improving Language Adherence in Multilingual LLMs
Multilingual Large Language Models (LLMs) develop cross-lingual abilities despite being trained on limited parallel data. However, they often struggle to generate responses in the intended language, favoring high-resourc…
CoLACoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
The full-size MLPs and the projection layers in attention introduce tremendous model sizes of large language models (LLMs), imposing extremely demanding needs of computational resources in the pre-training stage. However…
CoLAOptimizing Language Models for Grammatical Acceptability: A Comparative Study of Fine-Tuning Techniques
This study explores the fine-tuning (FT) of the Open Pre-trained Transformer (OPT-125M) for grammatical acceptability tasks using the CoLA dataset. By comparing Vanilla-Fine-Tuning (VFT), Pattern-Based-Fine-Tuning (PBFT)…
CoLAComputational Efficiencyparameter-efficient fine-tuningPredicting Emergent Capabilities by Finetuning
A fundamental open challenge in modern LLM scaling is the lack of understanding around emergent capabilities. In particular, language model pretraining loss is known to be highly predictable as a function of compute. How…
CoLAGSM8KMMLUDARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models
Storing open-source fine-tuned models separately introduces redundancy and increases response times in applications utilizing multiple models. Delta-parameter pruning (DPP), particularly the random drop and rescale (DARE…
CoLAparameter-efficient fine-tuningTReX- Reusing Vision Transformer's Attention for Efficient Xbar-based Computing
Due to the high computation overhead of Vision Transformers (ViTs), In-memory Computing architectures are being researched towards energy-efficient deployment in edge-computing scenarios. Prior works have proposed effici…
CoLAEdge-computingImproving Fast Adversarial Training Paradigm: An Example Taxonomy Perspective
While adversarial training is an effective defense method against adversarial attacks, it notably increases the training cost. To this end, fast adversarial training (FAT) is presented for efficient training and has beco…
CoLACan VLMs be used on videos for action recognition? LLMs are Visual Reasoning Coordinators
Recent advancements have introduced multiple vision-language models (VLMs) demonstrating impressive commonsense reasoning across various domains. Despite their individual capabilities, the potential of synergizing these …
Action RecognitionCoLALanguage ModelingLanguage Modelling+2Empowering Persian LLMs for Instruction Following: A Novel Dataset and Training Approach
Instruction-tuned large language models have demonstrated remarkable capabilities in following human instructions across various domains. However, their proficiency remains notably deficient in many low-resource language…
CoLAInstruction FollowingADMM Based Semi-Structured Pattern Pruning Framework For Transformer
NLP(natural language processsing) has achieved great success through the transformer model.However, the model has hundreds of millions or billions parameters,which is huge burden for its deployment on personal computer o…
CoLAQuantizationRTECoLA: Conditional Dropout and Language-driven Robust Dual-modal Salient Object Detection
The depth/thermal information is beneficial for detecting salient object with conventional RGB images. However, in dual-modal salient object detection (SOD) model, the robustness against noisy inputs and modality missing…
CoLALanguage ModelingLanguage Modellingobject-detection+5CoLa-DCE -- Concept-guided Latent Diffusion Counterfactual Explanations
Recent advancements in generative AI have introduced novel prospects and practical implementations. Especially diffusion models show their strength in generating diverse and, at the same time, realistic features, positio…
CoLAcounterfactualimage-classificationImage ClassificationAn Information Theoretic Evaluation Metric For Strong Unlearning
Machine unlearning (MU) aims to remove the influence of specific data from trained models, addressing privacy concerns and ensuring compliance with regulations such as the "right to be forgotten." Evaluating strong unlea…
CoLAMachine Unlearningvalid