Low-resource domain adaptation while minimizing energy and hardware resource consumption
Training Large Language Models (LLMs) is costly in terms of energy, hardware, and annotated data, often resulting in a positionality rooted in predominant cultures and values (Santy et al., 2023). Domain adaptation has emerged as a promising strategy to better align models with diverse cultural and value contexts (Hershcovich et al., 2022), but its computational cost remains a significant barrier, particularly for research groups lacking access to large-scale infrastructure. In this paper, we evaluate how the use of different numerical precision formats and data parallelization strategies impacts both training speed (as a proxy to energy and hardware consumption) and model accuracy, with the goal of facilitating domain adaptation in low-resource environments. Our findings are relevant to any setting where energy efficiency, accessibility, or limited hardware availability are key concerns.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ALERT: Accurate Learning for Energy and Timeliness
An increasing number of software applications incorporate runtime Deep Neural Networks (DNNs) to process sensor data and return inference results to humans. Effective deployment of DNNs in these interactive scenarios req…
CPUGPUImage ClassificationDomain Borders Are There to Be Crossed With Federated Few-Shot Adaptation
Federated Learning has emerged as a leading paradigm for decentralized, privacy-preserving learning, particularly relevant in the era of interconnected edge devices equipped with sensors. However, the practical implement…
Domain AdaptationFederated LearningPrivacy PreservingOn-demand Test-time Adaptation for Edge Devices
Continual Test-time adaptation (CTTA) continuously adapts the deployed model on every incoming batch of data. While achieving optimal accuracy, existing CTTA approaches present poor real-world applicability on resource-c…
Test-time AdaptationORIENT: A Priority-Aware Energy-Efficient Approach for Latency-Sensitive Applications in 6G
Anticipation for 6G's arrival comes with growing concerns about increased energy consumption in computing and networking. The expected surge in connected devices and resource-demanding applications presents unprecedented…
Q-LearningCross-Domain Energy-Guided Diffusion Generation for Off-Dynamics Reinforcement Learning
Off-dynamics offline reinforcement learning seeks to learn a target-domain policy from a large source dataset and a limited target dataset under mismatched transition dynamics. Existing approaches such as reward augmenta…
Reinforcement LearningOffline RL