홈 › Papers

ExecuTorch -- A Unified PyTorch Solution to Run AI Models On-Device

2026-05-05 · Mergen Nachin, Digant Desai, Sicheng Stephen Jia, Chen Lai, Mengwei Liu, Jacob Szwejbka, Raziel Alvarez, RJ Ascani, Dave Bort, Manuel Candales, Andrew Caples, Yanan Cao, Zhengxu Chen, Soumith Chintala, Gregory Comer, Tanvir Islam, Songhao Jia, Tarun Karuturi, Jack Khuu, Abhinay Kukkadapu, Tugsbayasgalan Manlaibaatar, Andrew Or, Kimish Patel, Siddartha Pothapragada, Lucy Qiu, Supriya Rao, Orion Reblitz-Richardson, Max Ren, Scott Roy, Anthony Shoumikhin, Scott Wolchok, Guang Yang, Angela Yi, Martin Yuan, Hansong Zhang, Jack Zhang, Jerry Zhang, Shunting Zhang, C. Cagatay Bilgin arxiv

Local execution of AI on edge devices is important for low latency and offline operation. However, deploying models on diverse hardware remains fragmented, often requiring model conversion or complete reimplementation outside the PyTorch ecosystem where the model was originally authored. We introduce ExecuTorch, a unified PyTorch-native deployment framework for edge AI. ExecuTorch enables seamless deployment of machine learning models across heterogeneous compute environments. It scales from embedded microcontrollers to complex system-on-chips (SoCs) with dedicated accelerators, powering devices ranging from wearables and smartphones to large compute clusters. ExecuTorch preserves PyTorch semantics while allowing customization, support for optimizations like quantization, and pluggable execution "backends". These features together enable fast experimentation, allowing researchers to validate deployment behavior entirely within PyTorch, bridging the gap between research and production.

📄 PDF Abstract BibTeX arXiv:2605.08195

Code (0)

등록된 구현이 없습니다.

TorchAO: PyTorch-Native Training-to-Serving Model Optimization

2025-07-21 · Andrew Or, Apurva Jain, Daniel Vega-Myhre, Jesse Cai 외 arxiv

We present TorchAO, a PyTorch-native model optimization framework leveraging quantization and sparsity to provide an end-to-end, training-to-serving workflow for AI models. TorchAO supports a variety of popular model opt…

Enabling Efficient On-Device Fine-Tuning of LLMs Using Only Inference Engines

2024-09-23 · Lei Gao, Amir Ziashahabi, Yue Niu, Salman Avestimehr 외

Large Language Models (LLMs) are currently pre-trained and fine-tuned on large cloud servers. The next frontier is LLM personalization, where a foundation model can be fine-tuned with user/task-specific data. Given the s…

parameter-efficient fine-tuning Text Generation

A Scalable PyTorch Abstraction for Multi-GPU Gaussian Splatting

2026-06-09 · Matthew Cong, Francis Williams, Jonathan Swartz, Mark Harris 외 arxiv

Gaussian splatting methods have become increasingly popular for neural reconstruction of the real world. However, they are often limited in scale and resolution due to compute and memory constraints. We present a multi-G…

MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection

2025-06-24 · Zhengxiang Huang, Chaoyue Niu, Zhaode Wang, Jiarui Xue 외

As the demand for on-device Large Language Model (LLM) inference grows, energy efficiency has become a major concern, especially for battery-limited mobile devices. Our analysis shows that the memory-bound LLM decode pha…

CPU Large Language Model

Production-Grade Local LLM Inference on Apple Silicon: A Comparative Study of MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS

2025-10-09 · Varun Rajesh, Om Jodhpurkar, Pooja Anbuselvan, Mantinder Singh 외 arxiv

We present a systematic, empirical evaluation of five local large language model (LLM) runtimes on Apple Silicon: MLX, MLC-LLM, llama.cpp, Ollama, and PyTorch MPS. Experiments were conducted on a Mac Studio equipped with…