LG AI Research将K-EXAONE扩容三倍,推出7500亿参数的开放权重模型K-EXAONE 2.0
LG AI Research没有从零重新训练,而是把韩国政府支持项目下开发的K-EXAONE模型扩展架构后继续训练,得到总参数7500亿、每个token激活约370亿参数的混合专家(MoE)模型K-EXAONE 2.0,容量是前代的三倍以上。该模型支持最长25.6万token的上下文和10种语言,在自主编码代理和长上下文理解方面提升最大,在长上下文检索和安全性方面相较同类开放权重模型优势明显。模型以Apache 2.0许可发布,供更广泛的AI生态评估、部署与二次开发。
Figure 1: The main evaluation results of K-EXAONE 2.0.
Table 1: Model configurations of K-EXAONE 2.0 and K-EXAONE.
Block
Configuration
K-EXAONE 2.0
K-EXAONE
Main Block
Layers (Total/SWA/GA)
78 / 58 / 20
48 / 36 / 12
Sliding Window Size
128
128
Attention Heads (Q/KV)
64 / 8
64 / 8
Head Dimensions
128
128
Experts (Total/Shared/Activated)
256 / 1 / 8
128 / 1 / 8
Experts Dimensions
2,048
2,048
Parameters (Total/Activated)
750B / 37B
236B / 23B
MTP Block
Layers (Total/SWA/GA)
1 / 1 / 0
1 / 0 / 1
Attention Heads (Q/KV)
64 / 8
64 / 8
Head Dimensions
128
128
Parameters
0.52B
0.52B
DSpark Block
Layers (Total/SWA/GA)
5 / 0 / 5
–
Attention Heads (Q/KV)
64 / 8
–
Block Size (γ)
7
–
Parameters
2.53B
–
Figure 2: An illustration of K-EXAONE 2.0 model architecture. (Left): Main Model. (Right): MTP and DSpark modules. Incorporating either the MTP or DSpark module into the main model accelerates sequence generation. The main model consists of two initial dense layers followed by Mixture-of-Experts (MoE) layers. The sliding window attention (SWA) in the second layer employs a window size of 4096, whereas a window size of 128 is used for all other SWA layers. In the Sparse MoE layers, 8 routed experts are selected from a pool of 256 experts and deployed alongside one shared expert. To ensure stability during both training and inference, the last 16 layers of the main model apply Clamped SwiGLU with a limit value of 7.0 to the experts.
Table 2: Acceptance length and end-to-end speedup over non-speculative decoding, on the same K-EXAONE 2.0 (FP8) target with the same draft budget (γ=7). Each cell is non-thinking / thinking at temperature 1.0 on TP8, 8× H200.
Domain
Benchmark
Acceptance length
E2E speedup
MTP
DSpark
MTP
DSpark
Math
GSM8K
3.58 / 3.13
5.25 / 5.20
1.72 / 1.55
2.49 / 2.56
MATH-500
3.60 / 3.16
4.95 / 4.58
1.76 / 1.55
2.44 / 2.28
AIME 2026
3.00 / 2.73
4.00 / 3.60
1.50 / 1.36
2.01 / 1.81
Code
HumanEval
3.67 / 2.61
5.41 / 3.81
1.77 / 1.30
2.57 / 1.92
MBPP
3.14 / 2.55
4.19 / 3.60
1.53 / 1.27
2.05 / 1.81
Figure 3: Pre-training loss on the Korean subset of the K-EXAONE 2.0 pre-training mixture versus training compute (6⋅Nactive⋅D, FLOPs, log scale) for the K-EXAONE and EXAONE 4.0 lineages. Dashed lines mark GLM-5.1 and GLM-5.2.
Table 3: Absolute score changes relative to the initial late-stage checkpoint of K-EXAONE. All models are trained independently from the same checkpoint. The baseline model is trained on 30B tokens from the original pre-training mixture, while the Active Reading and textbook-style models are each trained on a 40B-token mixture consisting of the same 30B-token baseline data and an additional 10B synthetic tokens generated from Wikipedia documents.
Method
ARC-C [7]
MMLU [18]
GSM8K [8]
HellaSwag [66]
Avg.
Baseline Dataset
+0.00
-0.25
+1.51
+0.59
+0.46
Active Reading
+1.54
-0.11
+1.21
+0.34
+0.75
Textbook-style
-0.42
+0.55
+1.52
-0.02
+0.41
Figure 4: Needle-in-a-Haystack (NIAH) retrieval accuracy of K-EXAONE 2.0 across context lengths of up to 256K tokens and varying needle positions. K-EXAONE 2.0 maintains perfect retrieval throughout the evaluated range.
Table 4: Performance comparison on Korean benchmark categories across different data sources.
Data Source
Culture & History
Knowledge & Reasoning
Avg.
Institution-sourced Data
68.23
46.47
57.35
Open-sourced Data
67.19
47.21
57.20
Figure 5: Illustration of preserved thinking.
Table 5: Absolute Humanity’s Last Exam score improvements obtained from different training paths starting from the Mid Stage 1 checkpoint of the small-scale model.
Training Stage
Δ HLE
Mid-Stage 1 → Base SFT
+3.15
Mid-Stage 1 → Mid-Stage 2
+4.71
Mid-Stage 1 → Mid-Stage 2 → Base SFT
+5.66
Figure 6: Demographics of the Safety Teacher Advisory Council.
Table 6: The main evaluation results of K-EXAONE 2.0 Reasoning mode. Asterisk (∗) indicates that the scores are from each baseline model’s official technical report, blog or leaderboard.
K-EXAONE 2.0 (Reasoning)
K-EXAONE (Reasoning)
Qwen3.5 (Reasoning)
GLM-5.1 (Reasoning)
DeepSeek V4 Pro (Reasoning: max)
Architecture
MoE
MoE
MoE
MoE
MoE
# Total Params
750B
236B
397B
754B
1.6T
# Activated Params
37B
23B
17B
40B
49B
World Knowledge
MMLU-Pro
83.5
83.8
89.8∗
86.0
87.5∗
GPQA-Diamond
82.2
79.1
88.4∗
86.2∗
90.1∗
Humanity’s Last Exam
18.3
13.6
† 28.7∗
31.0∗
37.7∗
Math
AIME 2026
92.3
92.2
91.3∗
95.3∗
95.2
HMMT Feb 2026
78.4
80.7
84.6
82.6∗
95.2∗
IMO-AnswerBench
78.6
76.3
80.9∗
83.8∗
89.8∗
Coding / Agentic Coding
SciCode
40.1
35.6
42.0∗
43.8∗
50.0∗
SWE-Bench Verified
68.2
49.4
76.4∗
73.6
80.6∗
Terminal-Bench 2.1
43.8
30.3
51.3∗
61.8∗
64.0∗
Agentic Tool Use
τ3-Banking
14.2
14.2
13.4∗
11.5∗
25.8∗
Claw-Eval (general)
80.0
74.3
81.2
86.1
83.5
Instruction Following
IFEval
92.4
89.7
92.6∗
93.9
94.0
IFBench
72.6
67.3
76.5∗
76.3∗
76.5∗
Long Context Understanding
OpenAI-MRCR
94.4
52.3
93.0
71.5
92.9
AA-LCR
56.2
53.5
65.7∗
62.3∗
66.3∗
Ko-LongBench
89.6
86.8
91.3
83.6
91.4
Korean
KMMLU-Pro
69.1
67.3
77.4
75.8
80.5
CLIcK
84.2
83.9
88.9
88.7
91.6
HRM8K-KSM
91.1
91.9
91.2
89.4
94.3
Multilinguality
Table 7: Multilingual performance comparison on POLYMATH.
KO
DE
ES
JA
VI
PT
FR
IT
K-EXAONE
55.5
59.3
57.8
58.2
56.9
57.3
58.7
55.2
K-EXAONE 2.0
68.8
70.3
70.4
73.6
69.5
71.2
74.0
72.6
Table 8: Multilingual performance comparison on GlobalMMLU-Lite.
KO
DE
ES
JA
VI
PT
FR
IT
PL
K-EXAONE
86.3
86.5
88.5
88.0
84.8
86.5
88.8
86.3
86.3
K-EXAONE 2.0
86.5
88.3
87.8
87.3
83.0
88.0
87.8
86.3
84.5
Table 10: Revision of K-AUT through newly discovered risks.
Domain
V1
V2
New
Universal Human Values
55
69
14
Social Safety
75
89
14
Korean Sensitivity
60
87
27
Future Risk
36
51
15
Total
226
296
70
Table 11: Safety performance comparison on KGC-Safety.
This technical report presents K-EXAONE 2.0, an open-weight multilingual foundation model developed by LG AI Research as a step in our effort toward global frontier-scale foundation models. Rather than training from scratch, we upcycle K-EXAONE and expand its architecture, yielding a Mixture-of-Experts (MoE) model with 750B total parameters and approximately 37B activated per token---more than three times the capacity of its predecessor. K-EXAONE 2.0 supports context lengths of up to 256K tokens and expands multilingual coverage from six to ten languages. Its training pipeline combines continual pre-training, difficulty-focused mid-training, and post-training to strengthen reasoning, agentic coding, multilingual capability, and safety grounded in Korean sociocultural contexts. Across nine evaluation categories selected to reflect the conditions of practical use, K-EXAONE 2.0 improves over K-EXAONE and remains competitive with open-weight models, showing its largest gains in agentic coding and long-context understanding and its clearest strengths in long-context retrieval and safety. Released under the Apache 2.0 license, K-EXAONE 2.0 enables the wider AI ecosystem to evaluate, deploy, adapt, and build upon it, while marking the beginning---rather than the endpoint---of our challenge toward the global frontier.
作者 · Eunbi Choi, Kibong Choi, Sehyun Chun, Seokhee Hong, Junwon Hwang, Hyojin Jeon, Ahra Jo, Hyunjik Jo, Yeonsik Jo, Minhyeok
