K-Culture GlossaryㅈTechnical words in the news
MoE (Mixture of Experts)
MoE
An architecture that wakes up only the 'experts' needed for each question — a trick for running huge models cheaply.
In plain words
Mixture of Experts (MoE) — the model has many 'expert' components inside it, but for each question, only a few of them get activated. The overall model is huge, but only a portion of it does the work at any given time, which is the trick behind getting big-model performance at a fraction of the compute cost.
Think of a hospital with 100 doctors, but each patient only sees the two or three specialists relevant to their case. DeepSeek and Mistral became well known for building 'cheap but powerful' models this way, and many of today's leading large models use this approach. When a release notes something like '600 billion total parameters, 37 billion active parameters,' that's a sign it's using MoE.
See also
Stories using this term
No story has used this term yet. New ones attach here automatically.