google-ai-edge/litert-samples
A Google-built cookbook for running AI models directly on phones and edge devices
This repository collects sample apps, model conversion recipes, and automation tools for LiteRT, Google's on-device machine learning framework, and LiteRT-LM, a layer for running large language models efficiently with LiteRT. It includes working end-to-end apps for things like speech recognition, image generation, and streaming text-to-speech, plus pipelines that convert PyTorch or Hugging Face models into a form that runs on-device. The project is written mainly in Python and has 391 stars.
What it does
- LiteRT (formerly TensorFlow Lite) runs machine learning models directly on devices like phones using GPU/NPU hardware acceleration; LiteRT-LM is a dedicated layer built on top of it for running large language models (LLMs) efficiently
- The repo is organized into four areas: samples (runnable apps), models (conversion recipes), utilities (shared tools), and skills (automated agent workflows)
- Recent additions include a tiny 15-million-parameter, 32 MB streaming text-to-speech app (KittenTTS nano), an automatic speech recognition (ASR) sample, a multimodal app called PhotoTalk that combines vision and audio/text generation, Qwen3-based speech models, and a text-to-image diffusion model called Bonsai Image 4B
- The skills folder offers a step-by-step lifecycle: converting a model to run cleanly on GPU, quantizing it (compressing to fp16/int8/int4) without losing accuracy, verifying it on real hardware, and scaffolding an Android app around it
- Samples run across Android, iOS, Python, and web/WebAssembly platforms, using either the newer CompiledModel API or the legacy Interpreter API
Why it matters
For developers who want models running on a user's device rather than a cloud server, this repo works as a practical reference covering the full path from conversion to optimization, verification, and app-building. Having ready-made speech, image, and LLM examples that use modern NPU/GPU acceleration makes it easier to start building on-device AI apps quickly.
Terms in this repo
- LiteRT · Google's framework for running machine learning models on-device, formerly known as TensorFlow Lite
- LiteRT-LM · A specialized layer built on LiteRT for efficiently running large language models
- CompiledModel API · A modern model-execution interface supporting GPU/NPU acceleration and asynchronous execution
- Interpreter API · An older, broadly compatible model-execution interface
- Quantization · A technique that reduces a model's numeric precision (e.g., fp16, int8, int4) to shrink size and computation
- NPU · A hardware chip specialized for accelerating neural network computation
Repository description (English)
LiteRT and LiteRT-LM sample apps, model recipes, agent skills and utilities.
Open on GitHubTrending repos
- vorssaint/vorssaint-utilsOne free menu bar app replaces a dozen paid Mac utilities
- Alishahryar1/free-claude-codeA local proxy that lets coding AI agents run on 49 free or cheap model providers instead of one paid service
- freestylefly/awesome-gpt-image-2A library of 532 reverse-engineered prompts that turn GPT-Image2 into a predictable image-making tool
- block/buzzAn open-source workspace where humans and AI agents chat, code, and review in the same rooms
- NousResearch/hermes-agentNous Research's Hermes is an AI agent that gets smarter the more you use it
- virgiliojr94/book-to-skillA tool that turns technical book PDFs into on-demand reference skills for AI coding agents
- VoltAgent/awesome-agent-skillsA single hub collecting over 1000 'how-to' manuals that make AI coding assistants act like experts
- anthropics/claude-plugins-communityA shared shelf where anyone's Claude add-ons get listed for install