microsoft/onnxruntime
Microsoft's engine that lets AI models built in PyTorch or TensorFlow run fast on almost any device
onnxruntime runs models built with various deep learning frameworks and classical machine learning libraries quickly across many types of hardware and operating systems. It speeds up inference by optimizing the model's computation graph and tapping into hardware acceleration where available. It also supports multi-GPU training acceleration for transformer models with just a one-line change to existing PyTorch training scripts.
What it does
- Supports models from deep learning frameworks like PyTorch and TensorFlow/Keras, as well as classical ML libraries such as scikit-learn, LightGBM, and XGBoost
- Boosts performance by optimizing the model's internal computation graph and automatically leveraging hardware-specific acceleration
- Can speed up multi-node NVIDIA GPU training for transformer models by adding just one line to existing PyTorch training scripts
- Designed to work consistently across different hardware, drivers, and operating systems
Why it matters
The environment where a model is deployed often differs from the one it was trained in, and onnxruntime bridges that gap to cut deployment cost and improve response speed. With over 21,000 GitHub stars, it's a widely used piece of infrastructure that directly affects anyone building and shipping AI-powered products.
Terms in this repo
- ONNX · An open format for representing models built in different deep learning frameworks in a common way
- inference · The process of feeding new data into a trained model to get a result
- graph optimization · Restructuring a model's sequence of computations so it produces the same result faster
- hardware accelerator · Specialized hardware like a GPU or NPU built to perform certain computations faster than a general CPU
Repository description (English)
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
Open on GitHubTrending repos
- vorssaint/vorssaint-utilsOne free menu bar app replaces a dozen paid Mac utilities
- Alishahryar1/free-claude-codeA local proxy that lets coding AI agents run on 49 free or cheap model providers instead of one paid service
- freestylefly/awesome-gpt-image-2A library of 532 reverse-engineered prompts that turn GPT-Image2 into a predictable image-making tool
- block/buzzAn open-source workspace where humans and AI agents chat, code, and review in the same rooms
- NousResearch/hermes-agentNous Research's Hermes is an AI agent that gets smarter the more you use it
- virgiliojr94/book-to-skillA tool that turns technical book PDFs into on-demand reference skills for AI coding agents
- VoltAgent/awesome-agent-skillsA single hub collecting over 1000 'how-to' manuals that make AI coding assistants act like experts
- anthropics/claude-plugins-communityA shared shelf where anyone's Claude add-ons get listed for install