Everything K-culture — comebacks to K-beauty, straight to your inboxGet it in your inbox›
K-EXAONE 2.0 Technical Report
arXiv:2608.045052026-08-06
LG AI Research scales up K-EXAONE into a 750B-parameter open-weight model, K-EXAONE 2.0
Instead of training from scratch, LG AI Research expanded its Korean-government-backed K-EXAONE model into a Mixture-of-Experts model with 750B total parameters and about 37B activated per token, more than triple its predecessor's capacity. K-EXAONE 2.0 supports contexts up to 256K tokens and ten languages, with its largest gains in agentic coding and long-context understanding, and clear strengths in long-context retrieval and safety compared to open-weight peers. It is released under the Apache 2.0 license for the wider community to evaluate, deploy, and build upon.
METAL MEDIA explanatory visual
From K-EXAONE to K-EXAONE 2.0: the upcycling pipeline
Evidence statusMeasured results and planned work
Base model: K-EXAONE236B total parameters, 48 layers, 128 experts, 6 languages, up to 256K-token context
Architecture expansion (upcycling)Layers grown to 78, experts doubled to 256; duplicated weights get rotation noise to break symmetry, stabilized with Clamped SwiGLU
Mid-training + post-trainingContext window extended from 8K to 64K to 256K tokens; online reinforcement learning and SFT focused on reasoning, agentic coding, and safety
Inference acceleration modulesMTP and DSpark speculative decoding paths speed up generation by up to 2.57x
Result: K-EXAONE 2.0750B total parameters, ~37B activated per token, 10 languages, largest gains in agentic coding, long-context understanding, and safety
An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.
What they did
Rather than retraining from scratch, the team 'upcycled' the existing K-EXAONE (236B total parameters) by expanding its architecture and continuing large-scale pre-training on top of the reused weights.
Layers grew from 48 to 78 and experts per layer doubled from 128 to 256; to keep training stable, the last 16 layers use 'Clamped SwiGLU', which caps internal activation values at a threshold of 7.0.
Difficulty-focused mid-training and post-training strengthened reasoning, agentic coding, multilingual ability, and safety, while the context window was extended in stages from 8K to 64K to 256K tokens.
Two speculative-decoding paths -- Multi-Token Prediction (MTP) and a new DSpark drafter, which draft multiple tokens ahead for verification -- were added to speed up generation.
Korea-specific safety criteria (K-AUT) were expanded from 226 to 296 risk areas, developed with input from a Safety Teacher Advisory Council of 46 UNESCO-trained teachers who surfaced previously undefined risks.
Figure 1: The main evaluation results of K-EXAONE 2.0.
Table 1: Model configurations of K-EXAONE 2.0 and K-EXAONE.
Block
Configuration
K-EXAONE 2.0
K-EXAONE
Main Block
Layers (Total/SWA/GA)
78 / 58 / 20
48 / 36 / 12
Sliding Window Size
128
128
Attention Heads (Q/KV)
64 / 8
64 / 8
Head Dimensions
128
128
Experts (Total/Shared/Activated)
256 / 1 / 8
128 / 1 / 8
Experts Dimensions
2,048
2,048
Parameters (Total/Activated)
750B / 37B
236B / 23B
MTP Block
Layers (Total/SWA/GA)
1 / 1 / 0
1 / 0 / 1
Attention Heads (Q/KV)
64 / 8
64 / 8
Head Dimensions
128
128
Parameters
0.52B
0.52B
DSpark Block
Layers (Total/SWA/GA)
5 / 0 / 5
–
Attention Heads (Q/KV)
64 / 8
–
Block Size (γ)
7
–
Parameters
2.53B
–
Figure 2: An illustration of K-EXAONE 2.0 model architecture. (Left): Main Model. (Right): MTP and DSpark modules. Incorporating either the MTP or DSpark module into the main model accelerates sequence generation. The main model consists of two initial dense layers followed by Mixture-of-Experts (MoE) layers. The sliding window attention (SWA) in the second layer employs a window size of 4096, whereas a window size of 128 is used for all other SWA layers. In the Sparse MoE layers, 8 routed experts are selected from a pool of 256 experts and deployed alongside one shared expert. To ensure stability during both training and inference, the last 16 layers of the main model apply Clamped SwiGLU with a limit value of 7.0 to the experts.
Table 2: Acceptance length and end-to-end speedup over non-speculative decoding, on the same K-EXAONE 2.0 (FP8) target with the same draft budget (γ=7). Each cell is non-thinking / thinking at temperature 1.0 on TP8, 8× H200.
Domain
Benchmark
Acceptance length
E2E speedup
MTP
DSpark
MTP
DSpark
Math
GSM8K
3.58 / 3.13
5.25 / 5.20
1.72 / 1.55
2.49 / 2.56
MATH-500
3.60 / 3.16
4.95 / 4.58
1.76 / 1.55
2.44 / 2.28
AIME 2026
3.00 / 2.73
4.00 / 3.60
1.50 / 1.36
2.01 / 1.81
Code
HumanEval
3.67 / 2.61
5.41 / 3.81
1.77 / 1.30
2.57 / 1.92
MBPP
3.14 / 2.55
4.19 / 3.60
1.53 / 1.27
2.05 / 1.81
Figure 3: Pre-training loss on the Korean subset of the K-EXAONE 2.0 pre-training mixture versus training compute (6⋅Nactive⋅D, FLOPs, log scale) for the K-EXAONE and EXAONE 4.0 lineages. Dashed lines mark GLM-5.1 and GLM-5.2.
Table 3: Absolute score changes relative to the initial late-stage checkpoint of K-EXAONE. All models are trained independently from the same checkpoint. The baseline model is trained on 30B tokens from the original pre-training mixture, while the Active Reading and textbook-style models are each trained on a 40B-token mixture consisting of the same 30B-token baseline data and an additional 10B synthetic tokens generated from Wikipedia documents.
Method
ARC-C [7]
MMLU [18]
GSM8K [8]
HellaSwag [66]
Avg.
Baseline Dataset
+0.00
-0.25
+1.51
+0.59
+0.46
Active Reading
+1.54
-0.11
+1.21
+0.34
+0.75
Textbook-style
-0.42
+0.55
+1.52
-0.02
+0.41
Figure 4: Needle-in-a-Haystack (NIAH) retrieval accuracy of K-EXAONE 2.0 across context lengths of up to 256K tokens and varying needle positions. K-EXAONE 2.0 maintains perfect retrieval throughout the evaluated range.
Table 4: Performance comparison on Korean benchmark categories across different data sources.
Data Source
Culture & History
Knowledge & Reasoning
Avg.
Institution-sourced Data
68.23
46.47
57.35
Open-sourced Data
67.19
47.21
57.20
Figure 5: Illustration of preserved thinking.
Table 5: Absolute Humanity’s Last Exam score improvements obtained from different training paths starting from the Mid Stage 1 checkpoint of the small-scale model.
Training Stage
Δ HLE
Mid-Stage 1 → Base SFT
+3.15
Mid-Stage 1 → Mid-Stage 2
+4.71
Mid-Stage 1 → Mid-Stage 2 → Base SFT
+5.66
Figure 6: Demographics of the Safety Teacher Advisory Council.
Table 6: The main evaluation results of K-EXAONE 2.0 Reasoning mode. Asterisk (∗) indicates that the scores are from each baseline model’s official technical report, blog or leaderboard.
K-EXAONE 2.0 (Reasoning)
K-EXAONE (Reasoning)
Qwen3.5 (Reasoning)
GLM-5.1 (Reasoning)
DeepSeek V4 Pro (Reasoning: max)
Architecture
MoE
MoE
MoE
MoE
MoE
# Total Params
750B
236B
397B
754B
1.6T
# Activated Params
37B
23B
17B
40B
49B
World Knowledge
MMLU-Pro
83.5
83.8
89.8∗
86.0
87.5∗
GPQA-Diamond
82.2
79.1
88.4∗
86.2∗
90.1∗
Humanity’s Last Exam
18.3
13.6
† 28.7∗
31.0∗
37.7∗
Math
AIME 2026
92.3
92.2
91.3∗
95.3∗
95.2
HMMT Feb 2026
78.4
80.7
84.6
82.6∗
95.2∗
IMO-AnswerBench
78.6
76.3
80.9∗
83.8∗
89.8∗
Coding / Agentic Coding
SciCode
40.1
35.6
42.0∗
43.8∗
50.0∗
SWE-Bench Verified
68.2
49.4
76.4∗
73.6
80.6∗
Terminal-Bench 2.1
43.8
30.3
51.3∗
61.8∗
64.0∗
Agentic Tool Use
τ3-Banking
14.2
14.2
13.4∗
11.5∗
25.8∗
Claw-Eval (general)
80.0
74.3
81.2
86.1
83.5
Instruction Following
IFEval
92.4
89.7
92.6∗
93.9
94.0
IFBench
72.6
67.3
76.5∗
76.3∗
76.5∗
Long Context Understanding
OpenAI-MRCR
94.4
52.3
93.0
71.5
92.9
AA-LCR
56.2
53.5
65.7∗
62.3∗
66.3∗
Ko-LongBench
89.6
86.8
91.3
83.6
91.4
Korean
KMMLU-Pro
69.1
67.3
77.4
75.8
80.5
CLIcK
84.2
83.9
88.9
88.7
91.6
HRM8K-KSM
91.1
91.9
91.2
89.4
94.3
Multilinguality
Table 7: Multilingual performance comparison on POLYMATH.
KO
DE
ES
JA
VI
PT
FR
IT
K-EXAONE
55.5
59.3
57.8
58.2
56.9
57.3
58.7
55.2
K-EXAONE 2.0
68.8
70.3
70.4
73.6
69.5
71.2
74.0
72.6
Table 8: Multilingual performance comparison on GlobalMMLU-Lite.
KO
DE
ES
JA
VI
PT
FR
IT
PL
K-EXAONE
86.3
86.5
88.5
88.0
84.8
86.5
88.8
86.3
86.3
K-EXAONE 2.0
86.5
88.3
87.8
87.3
83.0
88.0
87.8
86.3
84.5
Table 10: Revision of K-AUT through newly discovered risks.
Domain
V1
V2
New
Universal Human Values
55
69
14
Social Safety
75
89
14
Korean Sensitivity
60
87
27
Future Risk
36
51
15
Total
226
296
70
Table 11: Safety performance comparison on KGC-Safety.
Model
Universal Human Values
Social Safety
Korean Sensitivity
Future Risk
Total
Qwen3.5-397B-A17B
95.8
96.8
85.5
86.7
92.0
GLM-5.1-754B-A40B
76.4
76.7
60.3
58.3
69.3
DeepSeek V4 Pro (Reasoning: max)
87.5
87.3
80.8
69.7
82.8
EXAONE 4.0 32B
63.6
57.2
60.7
46.7
58.0
K-EXAONE
97.5
96.9
94.3
95.0
96.1
K-EXAONE 2.0
100
99.9
99.3
100
99.8
Findings
Under the same draft budget (gamma=7), the DSpark drafter achieved 32-66% longer acceptance length than MTP, and end-to-end speedup rose from MTP's 1.27-1.77x to DSpark's 1.81-2.57x (measured on 8x H200 GPUs, TP8).
Active Reading synthetic data produced the largest average improvement over the initial checkpoint, driven mainly by gains on ARC-C, while textbook-style synthetic data performed best on MMLU and GSM8K.
Institution-sourced Korean cultural/historical data performed better on Korean culture and history benchmarks, while high-quality open-source Korean data achieved higher average scores on knowledge-intensive benchmarks.
Mid-Stage 2 improved HLE (an expert-level knowledge benchmark) by 4.71 points over Mid-Stage 1 on the small-scale model, exceeding the 3.15-point gain from applying SFT directly after Mid-Stage 1; applying the same SFT after Mid-Stage 2 added a further 0.95 points for a total 5.66-point gain.
K-EXAONE 2.0 achieved perfect Needle-in-a-Haystack retrieval accuracy across context lengths up to 256K tokens, and after iterative K-AUT criteria revisions, it showed consistently higher Safe Rates across all evaluated KGC-Safety dimensions than previous K-EXAONE models.
Where it can be used
Long-document or full-codebase analysis tasks requiring up to 256K tokens of context in a single pass
Agentic software development assistance involving multi-step tool use and code editing
Services requiring review or safety checks sensitive to Korean-language and cultural context
Multilingual service prototypes covering the ten supported languages, including Spanish, German, and Japanese
A candidate open-weight base model for research teams wanting to adapt and deploy a large model on their own infrastructure
Limits and open work
Most reported comparisons are relative improvements over the previous K-EXAONE or specific open-weight models, not claims of best-in-class performance across all benchmarks
The Korean-data compute-scaling comparison (Figure 3) relies heavily on undisclosed internal data, so comparisons with the GLM-5 lineage are presented only as external reference points, not direct baselines
The mid-training stage-by-stage analysis was conducted on a small-scale model rather than the full-scale deployed model
Detailed revisions and full quantitative results for the expanded safety taxonomy (K-AUT-V2) are limited to appendix tables
The authors explicitly identify scaling the generation and curation of advanced knowledge/reasoning data as future work not yet completed
Why it matters
A government-backed, large-scale open-weight model that approaches global frontier performance while being grounded in Korean-language and cultural safety norms broadens what the domestic AI ecosystem can build on and adapt. The 'upcycling' approach to scaling -- expanding an existing trained model rather than retraining from zero -- also offers a practical, resource-efficient path for growing large models.
Terms in this paper
Mixture-of-Experts (MoE) · A network design where only a subset of specialized sub-modules ('experts') is activated per input, keeping compute lower than the total parameter count suggests.
Upcycling · Reusing the weights of an already-trained model and expanding its architecture, then continuing training, instead of training a new model from scratch.
Speculative decoding · A technique where a small draft module predicts several tokens ahead, which the main model then verifies and accepts, speeding up text generation.
Clamped SwiGLU · A method that caps internal activation values in the network at a fixed limit (here, 7.0) to prevent them from growing uncontrollably large.
Needle-in-a-Haystack (NIAH) · A test that measures how accurately a model can find a specific piece of information hidden inside a very long document.
Original abstract (English)
This technical report presents K-EXAONE 2.0, an open-weight multilingual foundation model developed by LG AI Research as a step in our effort toward global frontier-scale foundation models. Rather than training from scratch, we upcycle K-EXAONE and expand its architecture, yielding a Mixture-of-Experts (MoE) model with 750B total parameters and approximately 37B activated per token---more than three times the capacity of its predecessor. K-EXAONE 2.0 supports context lengths of up to 256K tokens and expands multilingual coverage from six to ten languages. Its training pipeline combines continual pre-training, difficulty-focused mid-training, and post-training to strengthen reasoning, agentic coding, multilingual capability, and safety grounded in Korean sociocultural contexts. Across nine evaluation categories selected to reflect the conditions of practical use, K-EXAONE 2.0 improves over K-EXAONE and remains competitive with open-weight models, showing its largest gains in agentic coding and long-context understanding and its clearest strengths in long-context retrieval and safety. Released under the Apache 2.0 license, K-EXAONE 2.0 enables the wider AI ecosystem to evaluate, deploy, adapt, and build upon it, while marking the beginning---rather than the endpoint---of our challenge toward the global frontier.
Authors · Eunbi Choi, Kibong Choi, Sehyun Chun, Seokhee Hong, Junwon Hwang, Hyojin Jeon, Ahra Jo, Hyunjik Jo, Yeonsik Jo, Minhyeok
