Everything K-culture — comebacks to K-beauty, straight to your inboxGet it in your inbox

METAL MEDIA

ice-only/localai-rag-live2d

★ 21PythonMIT

An open-source project lets you run a Live2D anime character on your desktop that chats using a fully offline AI model and remembers things from your own documents

This project combines a locally-run quantized language model with a document-search feature and a Live2D anime character to build a desktop AI companion that works without sending any data to the cloud. Users can drop text, markdown, or PDF files into a folder and the assistant will automatically search them for answers, while conversations are saved as memory that persists across sessions. It also supports QQ messaging app integration alongside the desktop app.

What it does

  1. Runs a quantized version of the Qwen3.5-4B language model locally via llama.cpp, using GPU acceleration (Vulkan) or CPU only, so no internet connection or cloud API is required for chatting
  2. Implements RAG (Retrieval-Augmented Generation): documents placed in a docs folder are automatically indexed using a Chinese-language embedding model (BAAI/bge-small-zh-v1.5) and ChromaDB vector database, so the AI can look up relevant text before answering
  3. Displays a draggable, transparent, borderless Live2D character on the desktop built with PyQt5 and a web rendering component, giving the chatbot a visual anime-style persona
  4. Saves conversation history as JSON files, separated per user and per channel, and automatically trims older messages based on a token budget so the context doesn't overflow
  5. Supports multiple platforms through an adapter pattern, including a desktop mode and a QQ chat app connector via the OneBot v11 protocol

Why it matters

It's a practical template for anyone wanting to build a private, offline AI assistant with a custom visual personality without relying on cloud AI services or exposing personal documents. The modular structure (UI, model, memory, RAG, adapters) also makes it a reusable reference for adding your own chat platform or knowledge base.

Terms in this repo

  • RAG · 모델이 답하기 전에 관련 문서를 검색해서 참고하는 방식(Retrieval-Augmented Generation)
  • GGUF · 용량을 줄인(양자화) 로컬 실행용 모델 파일 형식
  • 임베딩(embedding) · 텍스트를 의미 기반 숫자 벡터로 바꿔 검색에 쓰는 기술
  • ChromaDB · 임베딩 벡터를 저장하고 유사도 검색을 해주는 벡터 데이터베이스
  • Live2D · 2D 그림을 부드럽게 움직이게 만드는 애니메이션 캐릭터 렌더링 기술

Repository description (English)

一个本地ai聊天模型localaiAgentWithRAG，让你的本地数据不上传云端，可以自己加入所需要的记忆文件等等，修改角色形象，记忆对话，记忆库允许自己导入或者对话后自动记录

Open on GitHub

Trending repos

All repos →

Latest from METAL MEDIA