HULK: An Energy Efficiency Benchmark Platform for Responsible Natural Language Processing
Computation-intensive pretrained models have been taking the lead of many natural language processing benchmarks such as GLUE. However, energy efficiency in the process of model training and inference becomes a critical bottleneck. We introduce HULK, a multi-task energy efficiency benchmarking platform for responsible natural language processing. With HULK, we compare pretrained models' energy efficiency from the perspectives of time and cost. Baseline benchmarking results are provided for further analysis. The fine-tuning efficiency of different pretrained models can differ a lot among different tasks and fewer parameter number does not necessarily imply better efficiency. We analyzed such phenomenon and demonstrate the method of comparing the multi-task efficiency of pretrained models. Our platform is available at https://sites.engineering.ucsb.edu/~xiyou/hulk/.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingSimilar Papers 제목 키워드 기반
Hulk: Graph Neural Networks for Optimizing Regionally Distributed Computing Systems
Large deep learning models have shown great potential for delivering exceptional results in various applications. However, the training process can be incredibly challenging due to the models' vast parameter sizes, often…
Distributed ComputingGraph Neural NetworkHulk: A Universal Knowledge Translator for Human-Centric Tasks
Human-centric perception tasks, e.g., pedestrian detection, skeleton-based action recognition, and pose estimation, have wide industrial applications, such as metaverse and sports analysis. There is a recent surge to dev…
3D Human Pose EstimationAction RecognitionHuman Mesh RecoveryHuman Part Segmentation+7Untangling Dense Knots by Learning Task-Relevant Keypoints
Untangling ropes, wires, and cables is a challenging task for robots due to the high-dimensional configuration space, visual homogeneity, self-occlusions, and complex dynamics. We consider dense (tight) knots that lack s…
Analysis and Knowledge Discovery from Sensors Data to Improve Energy Efficiency
Increases in energy prices and the global goal of mitigating CO2 emissions necessitate the development of intelligent Building Management Systems (BMS) that operate on an energy-efficient basis. Data Centers, buildings a…
energy managementManagementFairSense-AI: Responsible AI Meets Sustainability
In this paper, we introduce FairSense-AI: a multimodal framework designed to detect and mitigate bias in both text and images. By leveraging Large Language Models (LLMs) and Vision-Language Models (VLMs), FairSense-AI un…
Fairness