EnviroLLM: Resource Tracking and Optimization for Local AI
Large language models (LLMs) are increasingly deployed locally for privacy and accessibility, yet users lack tools to measure their resource usage, environmental impact, and efficiency metrics. This paper presents EnviroLLM, an open-source toolkit for tracking, benchmarking, and optimizing performance and energy consumption when running LLMs on personal devices. The system provides real-time process monitoring, benchmarking across multiple platforms (Ollama, LM Studio, vLLM, and OpenAI-compatible APIs), persistent storage with visualizations for longitudinal analysis, and personalized model and optimization recommendations. The system includes LLM-as-judge evaluations alongside energy and speed metrics, enabling users to assess quality-efficiency tradeoffs when testing models with custom prompts.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Fast and Resource-Efficient Object Tracking on Edge Devices: A Measurement Study
Object tracking is an important functionality of edge video analytic systems and services. Multi-object tracking (MOT) detects the moving objects and tracks their locations frame by frame as real scenes are being capture…
Multi-Object TrackingObjectObject TrackingLarge-displacement 3D Object Tracking with Hybrid Non-local Optimization
Optimization-based 3D object tracking is known to be precise and fast, but sensitive to large inter-frame displacements. In this paper we propose a fast and effective non-local 3D tracking method. Based on the observatio…
3D Object TrackingCPUObject TrackingOn the Performance of Gradient Tracking with Local Updates
We study the decentralized optimization problem where a network of $n$ agents seeks to minimize the average of a set of heterogeneous non-convex cost functions distributedly. State-of-the-art decentralized algorithms lik…
Federated LearningA Proposed Artificial intelligence Model for Real-Time Human Action Localization and Tracking
In recent years, artificial intelligence (AI) based on deep learning (DL) has sparked tremendous global interest. DL is widely used today and has expanded into various interesting areas. It is becoming more popular in cr…
Action DetectionAction LocalizationObjectobject-detection+3Threat-Based Resource Allocation Strategy for Target Tracking in a Cognitive Radar Network
Cognitive radar is developed to utilize the feedback of its operating environment obtained from a beam to make resource allocation decisions by solving optimization problems. Previous works focused on target tracking acc…