juxhinr/bindwidth
A calculator that figures out how many servers you actually need to run an LLM in-house, and whether that's cheaper than renting
Bindwidth is an open-source, browser-only calculator that sizes on-premises large language model deployments and estimates total cost of ownership. It identifies which of three technical limits — KV cache memory, combined prefill/decode serving capacity, or session concurrency ceiling — actually constrains a given setup, then prices hardware against that real bottleneck. It runs entirely client-side with no server, database, or telemetry, and compares owned hardware, sovereign rentals, and vendor subscriptions side by side.
What it does
- Treats human interactive users and always-on autonomous agents as structurally different workloads with separate peak-load math
- Uses six model architecture parameters — total and active parameters, bytes per parameter, KV cache size per token, accepted tokens per step, prefill/decode ratio — to determine GPU count and speed
- Calculates three independent constraints (memory, serving throughput, session ceiling) and flags whichever produces the maximum requirement as the true binding limit
- Tags every hardware and model data point with a confidence level (measured, published, community, estimated, etc.) so users can see what's verified versus guessed
- Built as pure JavaScript functions with no dependencies, backed by a pinned test suite so calculations stay reproducible
Why it matters
It reframes the common 'should we self-host our AI?' debate as a sizing problem rather than a pure budget question, since buying against the wrong constraint wastes capacity while missing the limit that will actually fail under load. This matters for any team weighing private, on-prem AI infrastructure against cloud subscriptions where data governance or cost control is a factor.
Terms in this repo
- KV cache · temporary memory a model keeps to remember prior conversation context while generating text
- prefill/decode · the two serving phases: processing the input prompt (prefill) versus generating output tokens one at a time (decode)
- TCO (total cost of ownership) · the full cost of running hardware, including purchase, electricity, and staff time, not just the sticker price
- MoE (mixture-of-experts) · a model architecture where only a subset of parameters activate per token, so size and speed are decoupled
- on-premises · running software on a company's own servers rather than in the cloud
Repository description (English)
Evidence-aware on-prem LLM inference sizing and TCO calculator
Open on GitHubTrending repos
- vorssaint/vorssaint-utilsOne free menu bar app replaces a dozen paid Mac utilities
- Alishahryar1/free-claude-codeA local proxy that lets coding AI agents run on 49 free or cheap model providers instead of one paid service
- freestylefly/awesome-gpt-image-2A library of 532 reverse-engineered prompts that turn GPT-Image2 into a predictable image-making tool
- block/buzzAn open-source workspace where humans and AI agents chat, code, and review in the same rooms
- NousResearch/hermes-agentNous Research's Hermes is an AI agent that gets smarter the more you use it
- virgiliojr94/book-to-skillA tool that turns technical book PDFs into on-demand reference skills for AI coding agents
- VoltAgent/awesome-agent-skillsA single hub collecting over 1000 'how-to' manuals that make AI coding assistants act like experts
- anthropics/claude-plugins-communityA shared shelf where anyone's Claude add-ons get listed for install