hardrave/NIGHTRUN
A USB stick that turns any PC into a dedicated LLM chat box, with no operating system involved
NIGHTRUN is a Rust program that runs directly on a PC's firmware and boots straight into an offline chatbot, skipping the operating system entirely. It loads a compressed AI model into memory, checks it for corruption while loading, then locks the disk so nothing can read or write to it again during the session. It has been tested to produce the exact same answers as llama.cpp, a well known reference LLM engine, to prove its math is correct.
What it does
- Built as a single UEFI application (the low-level program firmware runs before any OS) that draws its own screen, reads USB keyboard input, and runs the AI model using multiple CPU cores directly, without Linux, a kernel, or a network stack underneath.
- Supports three model families (Llama 3.2, Qwen3, Granite 4.1) at sizes from 1.3 to 2.4 GB, using compressed weight formats called quantization (Q8_0, Q4_K, Q6_K) that shrink model size while keeping most of its accuracy, run through hand-written fast math routines (AVX2/NEON) for the specific CPU chip type.
- Verifies correctness by comparing its output token-by-token against llama.cpp, a widely used LLM inference engine, for every supported model, and tests its text tokenizer against Hugging Face's official implementation so wording quirks don't creep into answers.
- Ships a cautious installer that refuses to overwrite a computer's main system disk, requires typing the exact device path to confirm before flashing a USB stick or SD card, and re-reads the written drive afterward to confirm it matches byte-for-byte.
- On an 8-core virtual machine (QEMU/KVM), Llama 3.2 1B reaches about 52-56 tokens per second while reading the prompt and about 20 tokens per second while writing its answer, with the whole boot-to-chat process taking 5.6 seconds; a real Raspberry Pi 5 runs a larger 3B model much slower at 3.0 tokens per second for now.
Why it matters
It shows a working example of an AI model running with nothing between it and the hardware, useful for people curious about ultra-minimal, offline, tamper-resistant AI devices. It's also presented as an experiment in how far an AI coding assistant can be pushed to build low-level systems software, not just typical web apps.
Terms in this repo
- UEFI · the firmware that starts a PC before any operating system loads
- no_std · Rust code written without relying on a full operating system's standard library
- GGUF · a common file format for storing compressed LLM model weights
- quantization (Q8_0, Q4_K, Q6_K) · shrinking a model's numbers to fewer bits to save memory and speed, with some accuracy tradeoff
- KV cache · memory that stores a model's past calculations so it doesn't redo them for every new word
Repository description (English)
Boot your PC straight into an LLM. Rust, UEFI-resident, no operating system underneath.
Open on GitHubTrending repos
- vorssaint/vorssaint-utilsOne free menu bar app replaces a dozen paid Mac utilities
- Alishahryar1/free-claude-codeA local proxy that lets coding AI agents run on 49 free or cheap model providers instead of one paid service
- freestylefly/awesome-gpt-image-2A library of 532 reverse-engineered prompts that turn GPT-Image2 into a predictable image-making tool
- block/buzzAn open-source workspace where humans and AI agents chat, code, and review in the same rooms
- NousResearch/hermes-agentNous Research's Hermes is an AI agent that gets smarter the more you use it
- virgiliojr94/book-to-skillA tool that turns technical book PDFs into on-demand reference skills for AI coding agents
- VoltAgent/awesome-agent-skillsA single hub collecting over 1000 'how-to' manuals that make AI coding assistants act like experts
- anthropics/claude-plugins-communityA shared shelf where anyone's Claude add-ons get listed for install