google-ai-edge/LiteRT-CLI
A single command-line tool to convert, shrink, and run edge AI models on any chip
LiteRT-CLI is a Google command-line tool that converts models like PyTorch into the LiteRT (formerly TFLite) format used on phones and desktops, shrinks them via quantization, pre-compiles them for specific chips, and runs, benchmarks, and visualizes them on real devices. It supports CPU, GPU, and NPU hardware across desktop, Android, and cloud platforms. It's still an early preview with limited platform and feature coverage.
What it does
- A single 'litert' command covers the whole model workflow: download, convert, quantize, compile, run, benchmark, and visualize
- It can download models from Hugging Face, convert PyTorch models into LiteRT format, shrink them with INT8/INT4 quantization, and ahead-of-time (AOT) compile them for specific chips like Qualcomm NPUs
- It can actually run models on CPU/GPU/NPU on Android devices or desktops, benchmark their latency, and visualize model structure using Model Explorer
- Large language models like Gemma can be run and benchmarked via the 'litert lm' command, and a skill file lets coding agents perform these tasks from natural-language prompts
- Currently 'compile' only works on Linux, some features are missing on macOS and Windows, and NPU support is limited to Qualcomm for now, with MediaTek and Google Tensor coming soon
Why it matters
Building and deploying edge AI models usually requires juggling separate tools for conversion, quantization, on-device testing, and benchmarking; this consolidates all of that into one CLI, speeding up development. Its integration with coding agents means optimization and deployment tasks can be automated through plain-language prompts, which is directly useful for practical workflows.
Terms in this repo
- LiteRT · Google's lightweight AI model format for mobile and edge devices, formerly called TFLite
- Quantization · A technique that reduces numeric precision (e.g., to INT8) to shrink model size and computation
- NPU · A dedicated chip optimized for neural network computation
- AOT compilation · Compiling a model in advance for specific hardware to reduce runtime startup delay
- Model Explorer · Google's tool for visualizing the internal structure of AI models as a graph
Repository description (English)
A convenient CLI to streamline LiteRT related development workflows, including converting, quantizing, compiling, managing, running, benchmarking and visualizing LiteRT (TFLite) models on various hardwares (CPU / GPU / NPU) across platforms (desktop, mobile or cloud).
Open on GitHubTrending repos
- vorssaint/vorssaint-utilsOne free menu bar app replaces a dozen paid Mac utilities
- Alishahryar1/free-claude-codeA local proxy that lets coding AI agents run on 49 free or cheap model providers instead of one paid service
- freestylefly/awesome-gpt-image-2A library of 532 reverse-engineered prompts that turn GPT-Image2 into a predictable image-making tool
- block/buzzAn open-source workspace where humans and AI agents chat, code, and review in the same rooms
- NousResearch/hermes-agentNous Research's Hermes is an AI agent that gets smarter the more you use it
- virgiliojr94/book-to-skillA tool that turns technical book PDFs into on-demand reference skills for AI coding agents
- VoltAgent/awesome-agent-skillsA single hub collecting over 1000 'how-to' manuals that make AI coding assistants act like experts
- anthropics/claude-plugins-communityA shared shelf where anyone's Claude add-ons get listed for install