google-ai-edge/litert-torch
Google's tool converts PyTorch models to run directly on phones and IoT devices
litert-torch is a Python library from Google that converts PyTorch models into the .tflite format used by LiteRT, Google's on-device inference runtime. It builds on PyTorch's torch.export() feature and covers a broad set of Core ATen operators, running mainly on CPU today with early GPU and NPU support. It also includes a separate Generative API for authoring and quantizing transformer-based LLMs for on-device deployment.
What it does
- Converts a PyTorch model (e.g., resnet18) into a .tflite file with a few lines of code using litert_torch.convert()
- Works by tracing the model with PyTorch's torch.export() and mapping it to Core ATen operators, then serializing to LiteRT's flatbuffer format
- The converted model can run on Android, iOS, and IoT devices via LiteRT's compiled model API in C++ or Java, using CPU, GPU, or NPU acceleration
- A separate Generative API lets developers author and quantize transformer/LLM models natively in PyTorch, convert them, and package them with a tokenizer into a .litertlm file for deployment via LiteRT-LM
- The PyTorch converter is labeled Beta while the Generative API is labeled Alpha; requires Python 3.10–3.13, Linux, PyTorch >=2.4.0, and tf-nightly
Why it matters
For developers wanting to ship PyTorch-trained models to mobile or embedded devices, this removes the need to manually rewrite models in a different framework. It's especially relevant now as on-device LLM deployment becomes more common, letting teams stay in the PyTorch ecosystem while targeting edge hardware acceleration.
Terms in this repo
- LiteRT · Google's runtime for running AI models directly on devices like phones, replacing what was formerly called TensorFlow Lite
- .tflite · A compact file format for storing models so they can run efficiently on-device
- torch.export() · A PyTorch feature that captures a model's computation graph so it can be converted to other formats
- Core ATen operators · A standardized, smaller set of basic math operations that PyTorch models can be broken down into for portability
- NPU · Neural Processing Unit, a chip specialized for running AI computations efficiently on-device
- LiteRT-LM · A companion system for running large language models that have been converted and packaged for on-device use
Repository description (English)
Support PyTorch model conversion with LiteRT.
Open on GitHubTrending repos
- vorssaint/vorssaint-utilsOne free menu bar app replaces a dozen paid Mac utilities
- Alishahryar1/free-claude-codeA local proxy that lets coding AI agents run on 49 free or cheap model providers instead of one paid service
- freestylefly/awesome-gpt-image-2A library of 532 reverse-engineered prompts that turn GPT-Image2 into a predictable image-making tool
- block/buzzAn open-source workspace where humans and AI agents chat, code, and review in the same rooms
- NousResearch/hermes-agentNous Research's Hermes is an AI agent that gets smarter the more you use it
- virgiliojr94/book-to-skillA tool that turns technical book PDFs into on-demand reference skills for AI coding agents
- VoltAgent/awesome-agent-skillsA single hub collecting over 1000 'how-to' manuals that make AI coding assistants act like experts
- anthropics/claude-plugins-communityA shared shelf where anyone's Claude add-ons get listed for install