google-ai-edge/LiteRT
Google's on-device AI runtime, the successor to TensorFlow Lite
LiteRT is Google's on-device machine learning runtime, formerly known as TensorFlow Lite. It lets phones, laptops and browsers run everything from image classification to large language models locally, tapping CPU, GPU and NPU (dedicated AI chip) acceleration. Models built in PyTorch, TensorFlow or Jax can be converted and optimized, then deployed via C++, Kotlin or JavaScript APIs.
What it does
- As the successor to TensorFlow Lite, LiteRT introduces a new Compiled Model API that automatically picks the right accelerator (GPU/NPU) and supports true asynchronous execution, replacing the old interpreter-based approach.
- LiteRT-LM enables running large language models directly on-device, while LiteRT.js brings secure client-side inference to web browsers via WebGPU and WASM.
- It targets a wide range of platforms including Android, iOS, Linux, macOS, Windows, Web and IoT, offering a single API to access NPUs from vendors like Broadcom, Qualcomm, MediaTek and Intel.
- The pipeline converts PyTorch models with LiteRT Torch Converter, compresses them via the AI-Edge Quantizer, and deploys them in .tflite or .litertlm formats.
- Official guidance directs developers to use the new Compiled Model API instead of the legacy tflite::Interpreter, noting that classic TensorFlow Lite code is now in maintenance mode receiving only security and stability updates.
Why it matters
This matters for developers who need to run AI features like speech recognition, image classification or chatbots quickly and privately on devices without relying on the cloud. Unifying access to NPUs from multiple chip vendors under one API also significantly reduces development complexity.
Terms in this repo
- on-device · Running computation locally on a device like a phone or PC rather than on a remote server
- NPU · Neural Processing Unit, a chip dedicated to accelerating neural network computations
- Compiled Model API · LiteRT's new execution interface that automates accelerator selection and asynchronous execution
- LLM · Large Language Model, the large-scale AI models used in chatbots and similar systems
- WASM · WebAssembly, a technology that lets browsers run code at near-native speed
Repository description (English)
LiteRT, successor to TensorFlow Lite. is Google's On-device framework for high-performance ML & GenAI deployment on edge platforms, via efficient conversion, runtime, and optimization
Open on GitHubTrending repos
- vorssaint/vorssaint-utilsOne free menu bar app replaces a dozen paid Mac utilities
- Alishahryar1/free-claude-codeA local proxy that lets coding AI agents run on 49 free or cheap model providers instead of one paid service
- freestylefly/awesome-gpt-image-2A library of 532 reverse-engineered prompts that turn GPT-Image2 into a predictable image-making tool
- block/buzzAn open-source workspace where humans and AI agents chat, code, and review in the same rooms
- NousResearch/hermes-agentNous Research's Hermes is an AI agent that gets smarter the more you use it
- virgiliojr94/book-to-skillA tool that turns technical book PDFs into on-demand reference skills for AI coding agents
- VoltAgent/awesome-agent-skillsA single hub collecting over 1000 'how-to' manuals that make AI coding assistants act like experts
- anthropics/claude-plugins-communityA shared shelf where anyone's Claude add-ons get listed for install