정답 번역문 없이도 강화학습으로 번역 AI 품질을 올린 오픈소스 다국어 번역 모델 이야기

arXiv:2608.108122026-08-12

Reference-Free Post-Training of Open Large Language Models for Multilingual Machine Translation

Xiaomi 연구팀이 46개 언어를 지원하는 오픈 언어모델 MiLMMT-46을 대상으로, 정답 번역문 없이 두 개의 품질 평가 모델과 언어 판별기를 보상으로 쓰는 강화학습(GRPO)을 적용했다. 여기에 지도학습 체크포인트와 강화학습 체크포인트를 절반씩 섞는 간단한 방법을 더해 MiLMMT-46-v1.0을 만들었더니, Google Translate·Gemini 3 Pro·GPT-5 등 상용 시스템과 견줄 만한 참조 없는 품질 점수를 기록했다. 저자들은 더 작은 모델에 지식을 옮기는 '온폴리시 증류' 방법도 시도했지만, 이는 강화학습+체크포인트 혼합이 만든 품질 수준에 도달할 뿐 넘어서지는 못했다.

METAL MEDIA 해설 도표

MiLMMT-46 사후 학습 파이프라인

증거 상태측정 결과가 보고됨

출발점: MiLMMT-46-v0.1정답 번역문으로 지도학습된 46개 언어 지원 오픈 언어모델(1B/4B/12B)
보상 설계 및 GRPO 강화학습원문-번역 쌍에 XCOMET·COMETKiwi 점수를 평균 내고, 언어가 틀리면 보상을 깎는 언어 판별 게이트를 적용해 GRPO로 학습, MiLMMT-46-v0.1-RL 생성
체크포인트 보간지도학습 가중치와 강화학습 가중치를 α=0.5로 선형 혼합해 최종 MiLMMT-46-v1.0 생성
성능 비교WMT24++·FLORES+에서 Seed-X, HY-MT2, TranslateGemma 등 오픈 모델 및 Google Translate, Gemini 3 Pro, GPT-5와 참조 없는 점수 비교
온폴리시 증류 실험12B 모델을 교사로 삼아 1B·4B 학생 모델에 지식을 옮기는 대안 경로를 시험, 강화학습+보간이 만든 품질 한계에 도달하지만 넘지 못함

METAL MEDIA이 원문을 바탕으로 재구성한 해설 도표이며, 논문 저자의 원문 figure가 아닙니다.

무엇을 했나

번역 모델을 더 좋게 만들려면 보통 정답 번역문(참조문)이 있는 데이터로 학습하는데, 저자원 언어일수록 이런 데이터가 부족하다. 이 연구는 정답 번역문 없이 원문 문장만으로 학습하는 방법을 시도했다.
지도학습으로 만든 MiLMMT-46-v0.1 모델에서 시작해, GRPO라는 강화학습 방법으로 XCOMET과 COMETKiwi 두 품질 평가 모델의 평균 점수를 보상으로 주고, 언어가 틀리게 나오면 보상을 깎는 '언어 판별 게이트'를 추가했다.
강화학습만 하면 유창하지만 다른 언어로 번역하거나 어색한 표현이 늘어나는 부작용(보상 해킹)이 생길 수 있어, 지도학습 체크포인트와 강화학습 체크포인트를 절반씩(α=0.5) 섞는 '체크포인트 보간'으로 균형을 맞췄다.
1B, 4B, 12B 세 가지 크기 모델에 같은 방법을 적용한 결과, WMT24++ 벤치마크에서 참조 없는 XCOMET과 COMETKiwi 점수가 지도학습 모델 대비 평균 2.75점, 2.44점 올랐고, 최종 모델 MiLMMT-46-v1.0은 Seed-X, HY-MT2, TranslateGemma 같은 오픈소스 경쟁 모델과 Google Translate, Gemini 3 Pro, GPT-5 같은 상용 시스템 대비 참조 없는 평가에서 우수한 성적을 냈다.
더 작은 모델을 따로 강화학습하는 대신, 12B 모델이 배운 것을 1B·4B 학생 모델에 온폴리시 증류로 전달하는 실험도 진행했는데, 학생 모델은 강화학습+체크포인트 보간이 만든 품질 수준에 가깝게 도달했지만 이를 넘어서지는 못했다.

Figure 1: Training and validation rewards during GRPO post-training at the 1B, 4B, and 12B scales. Validation rewards are evaluated every 50 training steps.

Table 1: Translation performance on WMT24++ and FLORES+. WMT24++ QE cells report reference-free XCOMET / COMETKiwi. For FLORES+, Ref. cells report spBLEU / reference-based XCOMET, while QE cells report reference-free XCOMET / COMETKiwi. The best and second-best scores for each metric within each language-coverage block are shown in bold and underlined, respectively. Detailed reference-free and reference-based results are provided in Appendix D.1 and Appendix D.2, respectively.
Model	WMT24++	FLORES+
	en → xx	en → xx	xx → en	zh → xx	xx → zh
	QE	Ref.	QE	Ref.	QE	Ref.	QE	Ref.	QE
21 languages
Tower-Plus-2B	82.52/76.30	40.41/92.65	94.10/88.53	42.54/94.61	94.88/89.77	24.27/88.56	87.99/78.87	29.53/84.65	71.29/76.52
Tower-Plus-9B	86.80/80.76	43.33/95.09	95.93/91.05	45.32/96.03	95.84/90.44	27.59/92.65	90.73/80.68	33.25/88.09	73.95/78.59
Tower-Plus-72B	86.25/80.21	42.74/94.86	95.76/90.71	45.89/96.26	96.00/90.51	28.50/92.92	90.55/80.41	35.06/89.17	74.61/78.88
MiLMMT-46-1B-v1.0	83.28/76.69	36.87/92.02	93.88/88.03	41.10/94.45	95.15/89.73	22.94/87.49	87.57/78.29	28.20/85.39	74.03/77.78
MiLMMT-46-4B-v1.0	89.11/82.77	39.85/95.64	96.67/91.85	43.45/96.06	96.33/90.57	26.53/92.83	91.38/82.07	31.47/89.49	76.77/80.45
MiLMMT-46-12B-v1.0	90.77/84.55	40.66/96.60	97.39/92.88	44.59/96.55	96.53/90.77	28.03/94.43	92.28/83.14	32.19/90.79	78.01/81.43
26 languages
Seed-X-Instruct-7B	85.19/78.30	44.16/94.45	95.30/90.17	44.54/93.10	91.92/88.67	28.60/90.43	85.68/80.96	32.51/87.60	68.36/79.97
Seed-X-PPO-7B	86.18/79.99	45.48/95.02	95.95/91.10	44.12/95.92	95.73/90.51	29.65/92.80	89.32/83.88	29.20/88.86	68.90/80.88
MiLMMT-46-1B-v1.0	83.42/77.21	37.72/92.39	94.19/88.66	41.72/94.28	94.87/89.82	23.12/87.32	86.75/80.83	27.89/85.04	73.44/79.45
MiLMMT-46-4B-v1.0	89.19/83.25	40.50/95.87	96.86/92.40	44.30/95.94	96.06/90.68	26.72/92.75	90.63/84.79	31.36/89.32	76.30/82.31
MiLMMT-46-12B-v1.0	90.88/85.00	41.18/96.84	97.58/93.46	45.43/96.45	96.28/90.89	28.15/94.34	91.58/85.74	32.39/90.72	77.58/83.28
28 languages
GemmaX2-28-2B-v0.2	80.95/77.35	37.38/90.14	92.42/87.69	42.29/92.52	91.86/88.28	25.06/86.56	85.32/79.71	29.70/83.57	68.60/78.17
GemmaX2-28-9B-v0.2	84.19/80.28	40.04/92.66	94.32/89.56	44.53/94.02	92.88/88.81	27.67/90.27	87.94/81.75	32.39/86.42	70.41/79.78
MiLMMT-46-1B-v1.0	79.73/75.97	33.12/88.23	91.22/86.37	38.73/91.24	91.23/87.73	21.43/83.24	83.36/78.35	26.28/81.48	68.84/77.60
MiLMMT-46-4B-v1.0	86.13/81.81	36.16/93.19	95.05/90.30	42.05/93.86	93.09/88.75	25.01/90.03	88.47/82.29	30.11/86.55	71.54/80.32
MiLMMT-46-12B-v1.0	88.09/83.57	36.82/94.58	96.01/91.39	43.44/94.62	93.52/89.00	26.39/91.93	89.55/83.19	31.35/88.20	72.84/81.30
31 languages
HY-MT1.5-1.8B	83.88/77.38	24.81/88.51	91.80/85.31	25.25/88.51	87.51/84.01	17.60/82.79	84.93/74.74	22.61/82.58	70.06/77.06
Hunyuan-MT-7B	84.51/80.71	27.84/89.97	92.37/88.53	30.20/89.90	88.29/85.76	20.83/85.89	84.55/78.08	22.98/85.35	71.69/79.15
HY-MT1.5-7B	84.42/80.85	28.97/90.16	92.37/88.78	31.68/90.44	88.87/86.20	20.61/86.38	85.45/79.35	23.59/86.18	72.45/79.10
HY-MT2-1.8B	82.61/78.30	30.97/88.40	91.14/86.20	32.29/88.57	88.29/85.83	21.31/85.17	84.87/76.82	24.82/82.60	70.89/75.74
HY-MT2-7B	86.21/81.86	34.84/92.10	93.92/89.68	38.79/92.57	91.36/87.78	23.24/89.29	87.69/80.14	25.35/88.08	73.79/78.36
HY-MT2-30B-A3B	86.44/82.58	34.94/92.30	93.87/90.24	40.07/93.90	91.98/88.08	24.38/89.92	87.79/78.25	29.91/88.83	73.68/78.60
MiLMMT-46-1B-v1.0	78.95/75.94	32.54/87.82	90.76/86.40	37.95/91.06	90.80/87.37	21.89/83.36	83.74/77.14	26.64/81.85	69.44/76.82
MiLMMT-46-4B-v1.0	85.48/81.82	35.61/92.90	94.71/90.28	41.24/93.75	92.69/88.39	25.30/89.85	88.66/80.74	30.36/86.79	72.12/79.31

Figure 2: Trade-off between spBLEU and reference-based XCOMET under SFT–RL checkpoint interpolation on FLORES+, averaged over four translation-direction groups: en → xx, xx → en, zh → xx, and xx → zh. Each colored curve corresponds to one model scale, and α denotes the SFT weight. The MiLMMT-46-v0.1-RL and MiLMMT-46-v0.1 checkpoints form the two endpoints of each curve. Circled points indicate the selected MiLMMT-46-v1.0 checkpoints at α=0.5. External systems are included for comparison.

Table 2: On-policy distillation from MiLMMT-46-12B-v1.0 into 1B and 4B students, averaged over 46 languages. The v1.0 baseline is obtained through RL and checkpoint interpolation, and parenthetical labels indicate the student initialization. WMT24++ reports reference-free XCOMET, while FLORES+ reports spBLEU and reference-based XCOMET. RL + OPD adopts λ=1. Full per-group results and results for all tested values of λ are provided in Appendix E.
Model	WMT24++	FLORES+
1B
MiLMMT-46-1B-v1.0	79.01	30.51 / 85.94
OPD (init. v0.1)	77.67	30.47 / 85.22
RL + OPD (init. v0.1)	78.25	30.49 / 85.67
OPD (init. v1.0)	77.86	30.39 / 85.42
4B
MiLMMT-46-4B-v1.0	85.86	33.96 / 90.91
OPD (init. v0.1)	85.42	34.17 / 90.82
RL + OPD (init. v0.1)	85.85	33.98 / 90.98
OPD (init. v1.0)	85.37	34.16 / 90.79

Table 3: Number of instances for each translation direction in the RL training data, sorted by training count. All 192 directions are shown (30,572 training / 1,000 validation in total).
Direction	Train	Val	Direction	Train	Val	Direction	Train	Val
en → ru	2205	73	cs → ta	71	2	hu → es	32	1
en → cs	1776	68	pl → yue	68	4	zhs → hi	32	1
en → de	1479	47	hu → th	68	1	ur → ru	31	2
en → ja	1327	43	ta → sl	63	0	ro → pt	31	1
en → bn	1219	38	bn → ar	62	0	uz → kk	31	0
ja → en	1189	47	ru → tr	60	0	nb → fa	30	2
en → zhs	775	18	my → en	57	3	bg → sk	30	1
bn → en	726	19	hi → ta	56	1	cs → el	30	1
km → en	689	24	th → tr	55	2	it → th	30	1
ru → en	664	25	el → uz	53	4	lo → en	29	2
en → tr	644	22	fr → az	53	4	sl → da	29	2
de → en	620	17	sk → lo	53	1	da → ms	29	1
en → fr	612	15	el → km	53	0	fr → nb	29	1
en → he	598	18	fi → id	52	3	kk → nl	29	1
tr → en	510	22	az → tl	52	1	ca → bg	28	0
fr → de	463	17	hi → ms	51	0	bg → ro	27	1
en → pt	446	11	hr → tr	51	0	my → zht	27	0
de → fr	441	13	my → ur	50	3	ca → ru	25	0
en → uz	437	23	ur → bg	50	2	es → nl	25	0
cs → en	389	16	ur → hr	50	1	ms → sv	25	0
en → km	362	8	el → fi	49	4	fa → de	22	2
en → az	359	10	fi → ur	48	3	nb → bg	22	2
en → my	358	12	sk → ar	48	3	ru → es	22	1
en → it	336	12	it → tr	48	1	es → fr	22	0
en → tl	315	8	hr → zht	47	1	kk → ca	21	1
en → yue	306	11	bg → fi	47	0	pl → es	21	0
en → ur	300	13	ur → nl	46	0	es → bn	20	2
fi → en	300	12	az → km	45	3	el → ro	20	0
en → ta	294	5	tr → ca	45	2	fr → es	20	0
en → es	279	6	zht → kk	45	1	pt → ms	20	0

Table 4: 46 languages supported by our model. The resource of each language is determined according to the taxonomy classes by Joshi et al. (2020).
ISO Code	Language	Script	Family	Subgrouping	Resource
ar	Arabic	Arabic	Afro-Asiatic	Semitic	High
az	Azerbaijani	Latin	Turkic	Common Turkic	Low
bg	Bulgarian	Cyrillic	Indo-European	Balto-Slavic	Mid
bn	Bengali	Bengali	Indo-European	Indo-Aryan	Mid
ca	Catalan	Latin	Indo-European	Italic	High
cs	Czech	Latin	Indo-European	Balto-Slavic	High
da	Danish	Latin	Indo-European	Germanic	Mid
de	German	Latin	Indo-European	Germanic	High
el	Greek	Greek	Indo-European	Graeco-Phrygian	Mid
en	English	Latin	Indo-European	Germanic	High
es	Spanish	Latin	Indo-European	Italic	High
fa	Persian	Arabic	Indo-European	Iranian	High
fi	Finnish	Latin	Uralic	Finnic	High
fr	French	Latin	Indo-European	Italic	High
he	Hebrew	Hebrew	Afro-Asiatic	Semitic	Mid
hi	Hindi	Devanagari	Indo-European	Indo-Aryan	High
hr	Croatian	Latin	Indo-European	Balto-Slavic	High
hu	Hungarian	Latin	Uralic	-	High
id	Indonesian	Latin	Austronesian	Malayo-Polynesian	Mid
it	Italian	Latin	Indo-European	Italic	High
ja	Japanese	Japanese	Japonic	Japanesic	High
kk	Kazakh	Cyrillic	Turkic	Common Turkic	Mid
km	Khmer	Khmer	Austroasiatic	Khmeric	Low
ko	Korean	Hangul	Koreanic	Korean	High
lo	Lao	Lao	Tai-Kadai	Kam-Tai	Low
ms	Malay	Latin	Austronesian	Malayo-Polynesian	Mid
my	Burmese	Myanmar	Sino-Tibetan	Burmo-Qiangic	Low
nb	Norwegian	Latin	Indo-European	Germanic	Low
nl	Dutch	Latin	Indo-European	Germanic	High
pl	Polish	Latin	Indo-European	Balto-Slavic	High

Table 5: GRPO training hyper-parameters, shared across all three model scales.
Hyper-parameter	Value
Optimization
Actor learning rate	1e−6
Batch size (prompts)	128
PPO mini-batch size	128
Total epochs	3
KL loss type	low-variance KL
KL coefficient β	1e−3
Gradient checkpointing	Enabled
Rollout / sampling
Inference backend	vLLM
Group size G	8
Temperature	0.8
Top-p	0.95
Max prompt length	4096 tokens
Max response length	4096 tokens
Reward (reference-free)
Quality metrics	XCOMET, COMETKiwi
Language-matched reward	12(XCOMET+COMETKiwi)
Language-mismatch reward	0
Language identification	OpenLID-v3

Table 6: Reference-free evaluation results (XCOMET / COMETKiwi) of baseline models (Part I) on the WMT24++ benchmark.
Direction	TranslateGemma-4B	TranslateGemma-12B	TranslateGemma-27B	GemmaX2-28-2B	GemmaX2-28-9B	Hunyuan-MT-7B	HY-MT1.5-1.8B	HY-MT1.5-7B	HY-MT2-1.8B	HY-MT2-7B	HY-MT2-30B-A3B	Seed-X-Instruct-7B	Seed-X-PPO-7B	Tower-Plus-2B	Tower-Plus-9B	Tower-Plus-72B
en → ar	82.93 / 78.21	86.54 / 81.66	87.12 / 83.01	77.96 / 74.33	82.34 / 78.80	88.87 / 82.85	85.12 / 78.51	88.98 / 82.66	83.57 / 78.39	87.70 / 82.24	88.27 / 82.49	80.33 / 75.78	81.80 / 77.34	- / -	- / -	- / -
en → az	39.43 / 45.81	71.65 / 76.53	77.85 / 81.72	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → bg	83.19 / 77.11	88.78 / 83.66	89.77 / 84.98	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → bn	78.79 / 75.57	84.04 / 79.68	84.95 / 80.72	78.80 / 77.55	81.61 / 79.78	86.50 / 80.30	83.05 / 75.73	86.77 / 80.81	81.93 / 77.34	86.77 / 81.94	87.50 / 82.22	- / -	- / -	- / -	- / -	- / -
en → ca	79.59 / 71.97	85.65 / 78.91	86.59 / 80.12	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → cs	80.07 / 75.12	86.54 / 81.74	87.55 / 83.54	78.78 / 71.86	83.84 / 77.78	88.05 / 82.53	82.61 / 75.88	87.58 / 82.28	82.60 / 74.60	87.49 / 80.68	88.50 / 82.13	83.43 / 76.69	84.35 / 77.67	76.91 / 69.83	83.61 / 77.82	83.06 / 76.78
en → da	90.04 / 81.51	92.74 / 85.16	93.64 / 86.29	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	90.64 / 80.69	90.67 / 81.25	87.69 / 77.84	89.94 / 80.45	90.01 / 80.16
en → de	93.06 / 77.75	93.22 / 80.63	94.79 / 81.52	92.23 / 76.92	93.67 / 80.11	95.37 / 81.85	94.43 / 78.08	95.35 / 81.66	93.04 / 77.62	94.86 / 80.95	95.13 / 81.74	92.96 / 78.49	92.90 / 79.11	90.98 / 75.69	93.53 / 79.49	93.50 / 79.62
en → el	80.47 / 74.71	86.77 / 81.08	88.44 / 83.48	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → es	89.00 / 78.97	90.61 / 81.65	90.90 / 82.53	87.25 / 77.30	89.35 / 80.00	92.03 / 82.68	91.73 / 80.90	92.20 / 82.50	89.81 / 78.91	91.60 / 81.03	92.14 / 82.07	87.51 / 76.46	87.88 / 77.68	86.48 / 76.19	88.95 / 79.12	89.26 / 79.77
en → fa	79.77 / 77.50	85.90 / 83.49	87.56 / 84.94	78.73 / 78.25	83.25 / 82.22	86.53 / 82.36	82.33 / 77.46	86.45 / 82.69	81.30 / 78.27	86.63 / 83.59	87.53 / 84.72	- / -	- / -	- / -	- / -	- / -
en → fi	83.44 / 80.94	88.96 / 86.47	90.49 / 88.28	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	87.09 / 82.39	87.74 / 84.15	80.14 / 77.12	87.95 / 84.78	84.80 / 80.91
en → fr	84.52 / 78.40	87.37 / 81.47	87.44 / 82.54	83.77 / 76.44	86.09 / 79.03	88.73 / 82.09	87.31 / 78.83	88.97 / 82.00	86.28 / 78.09	87.67 / 79.95	89.02 / 81.71	85.68 / 77.51	85.42 / 78.28	81.46 / 74.36	86.28 / 79.80	86.85 / 79.88
en → he	76.75 / 71.40	84.05 / 79.29	86.31 / 81.14	78.41 / 73.43	82.23 / 77.52	78.48 / 70.66	76.35 / 68.93	78.34 / 71.17	76.90 / 70.31	84.75 / 78.57	85.67 / 79.95	- / -	- / -	- / -	- / -	- / -
en → hi	75.17 / 68.43	82.54 / 72.27	83.76 / 73.75	73.22 / 69.70	76.77 / 71.42	86.01 / 70.11	82.27 / 66.61	86.03 / 71.64	80.16 / 70.42	84.96 / 73.68	85.67 / 74.11	- / -	- / -	66.30 / 66.88	73.08 / 70.57	70.08 / 69.26
en → hr	80.35 / 75.45	87.39 / 82.84	89.60 / 85.46	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	82.19 / 75.46	83.71 / 77.94	- / -	- / -	- / -
en → hu	75.53 / 71.64	86.27 / 82.75	89.31 / 85.95	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	86.25 / 81.26	87.24 / 83.06	80.16 / 76.36	86.62 / 83.10	83.46 / 79.47
en → id	88.84 / 81.48	91.54 / 84.58	91.59 / 84.77	87.36 / 80.40	88.77 / 81.96	92.94 / 85.37	91.85 / 83.07	92.80 / 85.53	90.54 / 82.87	92.53 / 84.87	92.81 / 85.16	85.14 / 77.06	85.79 / 78.69	- / -	- / -	- / -
en → it	87.65 / 79.94	88.70 / 82.44	89.63 / 83.06	85.89 / 76.99	87.81 / 79.74	91.37 / 83.68	89.98 / 80.96	91.50 / 83.48	89.03 / 80.09	90.69 / 82.30	91.36 / 82.78	86.11 / 76.94	86.12 / 77.99	84.57 / 75.76	88.33 / 80.42	87.99 / 80.57
en → ja	83.02 / 84.36	87.23 / 87.40	88.47 / 88.05	84.31 / 85.03	86.62 / 86.65	90.91 / 88.98	88.71 / 86.96	90.83 / 88.90	87.95 / 86.93	90.69 / 88.78	90.92 / 88.80	83.15 / 82.85	85.19 / 84.73	83.05 / 84.49	86.60 / 86.88	87.39 / 87.18
en → kk	39.80 / 55.35	64.89 / 79.54	73.00 / 84.14	- / -	- / -	34.93 / 68.37	60.20 / 61.48	34.76 / 69.33	41.52 / 59.03	46.32 / 68.28	36.97 / 69.43	- / -	- / -	- / -	- / -	- / -
en → km	37.64 / 46.67	64.89 / 77.61	70.23 / 81.16	71.41 / 79.63	75.22 / 82.12	68.36 / 78.57	70.81 / 78.65	67.70 / 78.73	75.00 / 81.98	79.01 / 84.43	78.29 / 84.62	- / -	- / -	- / -	- / -	- / -
en → ko	85.97 / 82.51	89.88 / 85.85	90.64 / 86.69	84.02 / 82.15	88.07 / 85.49	92.21 / 87.24	89.95 / 84.97	91.77 / 87.06	88.47 / 84.94	91.96 / 87.14	91.74 / 87.23	79.96 / 77.72	83.58 / 81.42	81.83 / 80.54	86.78 / 84.44	87.09 / 84.32
en → lo	38.64 / 46.84	68.05 / 75.22	75.19 / 79.77	71.15 / 76.09	76.36 / 79.57	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → ms	83.00 / 76.45	88.75 / 82.31	89.23 / 82.52	85.13 / 78.27	86.58 / 79.42	91.58 / 83.08	90.82 / 80.83	91.44 / 83.35	89.17 / 81.03	90.72 / 82.57	91.10 / 83.08	81.27 / 73.07	82.98 / 75.48	- / -	- / -	- / -
en → my	39.71 / 56.00	55.85 / 75.06	63.73 / 79.88	66.36 / 78.74	73.53 / 82.35	62.45 / 76.34	66.14 / 77.59	62.74 / 76.23	71.04 / 81.55	75.00 / 83.04	76.80 / 84.68	- / -	- / -	- / -	- / -	- / -
en → nb	89.83 / 82.89	92.36 / 86.34	93.42 / 87.03	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	86.75 / 76.16	88.51 / 79.61	86.80 / 78.31	87.08 / 78.40	86.53 / 77.89
en → nl	90.10 / 80.01	92.89 / 84.14	93.67 / 85.03	89.74 / 79.56	92.24 / 82.94	92.60 / 82.34	90.91 / 79.30	92.25 / 82.37	89.46 / 79.20	93.26 / 83.24	94.06 / 84.81	91.43 / 80.74	91.88 / 82.37	88.66 / 78.27	91.68 / 82.18	91.86 / 82.59
en → pl	85.33 / 76.77	89.51 / 82.21	90.29 / 83.63	83.41 / 74.66	87.14 / 79.11	86.22 / 78.20	84.10 / 74.69	86.31 / 78.11	83.38 / 73.85	89.28 / 80.77	91.35 / 82.83	87.45 / 78.50	88.00 / 79.62	81.70 / 72.82	88.18 / 80.01	87.22 / 78.84
en → pt	89.87 / 81.24	91.26 / 83.63	91.57 / 84.42	88.22 / 79.29	90.19 / 81.81	92.56 / 84.15	91.50 / 81.97	92.59 / 84.48	90.67 / 81.01	91.44 / 82.21	92.20 / 83.28	87.36 / 77.40	88.11 / 79.23	86.20 / 77.25	89.02 / 80.76	89.26 / 81.47

Table 7: Reference-free evaluation results (XCOMET / COMETKiwi) of baseline models (Part II) and MiLMMT models on the WMT24++ benchmark.
Direction	Google Translate	NLLB-54.5B	Gemini 2.5 Pro	Gemini 3 Pro	GPT-5	MiLMMT-46-1B-v0.1	MiLMMT-46-1B-v1.0	MiLMMT-46-4B-v0.1	MiLMMT-46-4B-v1.0	MiLMMT-46-12B-v0.1	MiLMMT-46-12B-v1.0
en → ar	86.44 / 81.97	- / -	84.85 / 81.08	85.10 / 81.20	84.36 / 81.56	74.32 / 71.17	78.72 / 75.15	82.17 / 78.59	86.67 / 81.80	83.68 / 80.37	89.44 / 84.33
en → az	75.04 / 78.02	- / -	78.83 / 81.72	80.01 / 82.17	80.36 / 82.29	66.15 / 69.98	73.65 / 75.98	77.81 / 79.42	83.99 / 83.98	80.36 / 81.29	87.76 / 86.56
en → bg	85.07 / 80.04	- / -	86.98 / 81.51	87.13 / 81.84	87.24 / 82.49	78.31 / 72.59	83.44 / 77.26	85.18 / 79.44	89.97 / 84.08	87.18 / 81.96	91.83 / 86.83
en → bn	76.65 / 76.64	- / -	81.98 / 79.63	84.39 / 81.09	82.14 / 80.01	73.35 / 72.87	80.44 / 77.38	81.13 / 79.21	87.39 / 81.83	82.97 / 80.67	89.00 / 83.30
en → ca	82.82 / 76.02	- / -	84.16 / 77.08	84.97 / 77.73	85.39 / 78.61	79.25 / 70.51	82.62 / 74.97	84.88 / 77.31	88.83 / 81.39	85.82 / 78.49	90.60 / 83.50
en → cs	83.42 / 77.94	- / -	85.82 / 80.02	85.95 / 80.36	86.08 / 81.57	74.07 / 66.96	79.72 / 73.77	82.84 / 77.04	87.89 / 82.89	85.99 / 80.36	90.08 / 85.44
en → da	90.57 / 81.72	- / -	91.88 / 83.64	92.15 / 84.16	92.06 / 84.59	86.69 / 76.36	89.50 / 79.45	90.86 / 81.85	93.55 / 85.42	92.14 / 83.36	95.05 / 87.05
en → de	94.57 / 81.13	- / -	93.93 / 80.31	93.84 / 80.70	93.87 / 81.08	89.73 / 73.16	91.87 / 75.70	93.32 / 79.43	94.81 / 81.13	94.11 / 80.88	95.83 / 82.72
en → el	83.85 / 78.92	- / -	86.04 / 81.08	85.76 / 80.96	86.33 / 81.45	75.94 / 70.33	79.82 / 74.29	84.67 / 78.69	88.74 / 82.62	85.97 / 81.03	91.04 / 85.24
en → es	89.20 / 79.21	- / -	89.32 / 79.50	89.46 / 79.65	89.86 / 80.50	85.69 / 74.59	88.29 / 78.24	89.02 / 79.17	92.18 / 82.46	89.79 / 80.35	93.52 / 84.39
en → fa	82.95 / 82.21	- / -	84.28 / 82.93	84.95 / 83.20	84.81 / 83.55	70.87 / 69.98	77.74 / 76.46	80.84 / 80.03	86.74 / 83.31	83.88 / 82.99	89.83 / 86.07
en → fi	88.18 / 85.06	- / -	90.53 / 87.55	90.76 / 87.38	90.23 / 88.09	75.47 / 72.56	79.05 / 75.77	86.58 / 83.34	90.13 / 86.72	89.23 / 85.77	92.83 / 89.55
en → fr	88.37 / 80.72	- / -	85.92 / 78.89	85.78 / 78.70	86.18 / 79.68	80.72 / 73.15	83.88 / 76.74	85.41 / 78.14	89.12 / 81.89	86.72 / 79.98	90.87 / 83.71
en → he	80.72 / 76.43	- / -	82.68 / 78.61	83.26 / 79.20	83.42 / 79.48	72.15 / 66.61	75.51 / 70.09	81.07 / 76.50	86.10 / 80.69	83.11 / 78.75	88.98 / 83.05
en → hi	84.30 / 74.35	- / -	79.63 / 72.28	81.11 / 73.33	81.02 / 72.74	68.30 / 66.08	75.43 / 67.59	76.95 / 71.28	84.66 / 72.50	78.71 / 72.65	87.43 / 74.67
en → hr	85.80 / 82.13	- / -	87.48 / 82.91	87.74 / 83.33	87.97 / 83.58	78.57 / 73.02	83.74 / 77.81	86.04 / 81.20	90.68 / 85.53	87.88 / 82.84	92.33 / 87.30
en → hu	86.37 / 83.60	- / -	88.66 / 85.48	88.95 / 85.20	89.45 / 86.08	75.42 / 71.75	79.77 / 75.75	85.50 / 82.16	90.04 / 85.74	88.13 / 84.92	92.35 / 88.20
en → id	90.46 / 84.30	- / -	89.55 / 83.29	89.59 / 83.12	89.86 / 83.29	84.81 / 77.44	89.02 / 81.12	88.70 / 81.60	92.67 / 85.58	89.75 / 82.48	94.04 / 86.73
en → it	87.68 / 80.30	- / -	86.98 / 79.38	87.53 / 79.56	87.94 / 80.65	83.21 / 73.62	86.62 / 78.19	88.13 / 79.58	91.32 / 83.55	88.69 / 80.86	92.37 / 85.20
en → ja	87.60 / 87.49	- / -	87.16 / 86.92	87.42 / 87.37	88.01 / 87.90	79.56 / 81.91	84.57 / 84.91	86.25 / 86.28	90.24 / 88.89	88.38 / 87.63	90.80 / 89.57
en → kk	73.37 / 83.05	- / -	75.89 / 85.17	76.34 / 84.05	76.43 / 84.91	63.49 / 75.97	67.87 / 79.67	74.60 / 83.53	78.17 / 86.36	77.03 / 85.22	81.48 / 88.07
en → km	72.83 / 81.55	- / -	74.46 / 83.33	76.32 / 83.88	74.17 / 82.78	67.64 / 76.34	71.55 / 79.96	74.93 / 82.01	79.53 / 84.45	76.71 / 83.30	81.65 / 85.75
en → ko	90.76 / 86.91	- / -	88.69 / 85.01	89.34 / 85.89	88.85 / 86.06	80.71 / 79.32	85.25 / 82.66	87.27 / 84.30	90.46 / 86.54	89.07 / 86.01	92.43 / 88.02
en → lo	73.29 / 77.81	- / -	75.11 / 80.35	76.44 / 80.81	72.83 / 78.57	66.58 / 72.10	71.54 / 76.03	76.27 / 79.50	81.09 / 81.50	77.56 / 80.36	83.41 / 83.01
en → ms	82.57 / 76.26	- / -	86.49 / 80.14	86.24 / 79.79	86.75 / 80.42	83.15 / 75.98	87.19 / 79.63	86.21 / 79.31	91.89 / 84.13	87.10 / 80.22	93.51 / 85.64
en → my	70.34 / 80.67	- / -	72.74 / 83.01	73.61 / 83.07	72.24 / 82.92	59.62 / 73.65	66.31 / 77.34	72.25 / 82.11	78.40 / 84.23	75.30 / 83.95	80.98 / 85.58
en → nb	90.21 / 83.50	- / -	91.96 / 85.05	92.31 / 85.72	92.37 / 85.87	86.78 / 78.64	89.39 / 80.85	91.30 / 83.89	93.59 / 86.49	92.33 / 84.93	94.86 / 88.18
en → nl	93.72 / 85.21	- / -	92.18 / 83.21	92.14 / 83.43	92.42 / 84.25	87.32 / 76.36	89.77 / 79.31	91.46 / 81.90	93.62 / 83.98	92.75 / 83.27	94.82 / 85.92
en → pl	87.66 / 80.20	- / -	87.76 / 80.06	87.80 / 80.38	88.21 / 81.36	77.23 / 68.06	82.43 / 73.66	85.91 / 77.83	90.37 / 82.49	88.33 / 80.65	92.55 / 84.80
en → pt	91.25 / 82.85	- / -	89.28 / 80.68	89.21 / 80.75	89.84 / 81.82	86.88 / 77.10	89.58 / 80.34	89.53 / 81.25	92.72 / 84.68	90.89 / 82.48	93.76 / 86.38

Table 8: Reference-free English-centric evaluation results (XCOMET / COMETKiwi) of baseline models (Part I) on the FLORES+ benchmark.
Direction	TranslateGemma-4B	TranslateGemma-12B	TranslateGemma-27B	GemmaX2-28-2B	GemmaX2-28-9B	Hunyuan-MT-7B	HY-MT1.5-1.8B	HY-MT1.5-7B	HY-MT2-1.8B	HY-MT2-7B	HY-MT2-30B-A3B	Seed-X-Instruct-7B	Seed-X-PPO-7B	Tower-Plus-2B	Tower-Plus-9B	Tower-Plus-72B
en → ar	93.54 / 87.09	95.66 / 89.84	96.38 / 90.84	91.90 / 85.63	94.20 / 87.97	96.24 / 90.07	93.88 / 86.84	96.05 / 90.22	93.32 / 86.88	95.60 / 89.57	95.70 / 89.69	93.34 / 87.04	93.78 / 87.76	- / -	- / -	- / -
ar → en	87.46 / 84.55	90.50 / 85.89	91.93 / 86.59	91.31 / 86.44	92.65 / 87.19	90.88 / 85.58	88.41 / 83.29	91.20 / 85.75	89.88 / 85.05	92.30 / 86.81	92.62 / 86.69	90.08 / 85.88	92.70 / 86.99	- / -	- / -	- / -
en → az	50.49 / 54.71	87.63 / 87.30	92.72 / 91.02	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
az → en	67.08 / 83.64	76.62 / 86.34	81.93 / 87.87	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → bg	94.35 / 89.46	97.23 / 93.44	97.88 / 94.36	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
bg → en	91.52 / 89.06	93.40 / 90.01	94.93 / 90.79	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → bn	90.38 / 81.97	93.58 / 85.14	94.19 / 85.99	91.40 / 84.35	93.15 / 86.05	94.81 / 86.03	92.21 / 81.61	95.05 / 86.60	91.99 / 82.94	94.80 / 86.89	95.37 / 87.59	- / -	- / -	- / -	- / -	- / -
bn → en	83.27 / 82.49	87.26 / 84.17	89.70 / 85.15	88.99 / 85.76	90.71 / 86.67	83.57 / 81.19	85.63 / 80.88	85.57 / 82.73	85.82 / 83.39	88.70 / 85.33	90.36 / 86.00	- / -	- / -	- / -	- / -	- / -
en → ca	93.41 / 86.49	96.70 / 91.16	96.85 / 91.74	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
ca → en	91.23 / 90.53	93.42 / 91.84	94.83 / 92.26	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → cs	92.31 / 88.02	95.96 / 92.97	96.63 / 94.28	94.08 / 88.52	95.67 / 91.28	96.30 / 92.40	92.77 / 87.03	96.11 / 92.23	92.99 / 86.87	96.25 / 91.89	96.66 / 92.79	95.12 / 91.05	96.01 / 91.83	93.43 / 87.92	95.68 / 91.76	95.10 / 90.73
cs → en	92.65 / 90.36	93.93 / 91.39	95.19 / 91.99	95.39 / 91.77	96.12 / 92.16	94.12 / 90.84	92.91 / 88.80	93.99 / 90.86	94.15 / 90.32	95.79 / 92.02	95.67 / 91.85	91.50 / 90.54	95.85 / 92.22	94.65 / 91.33	95.51 / 92.23	95.83 / 92.13
en → da	96.86 / 91.47	98.21 / 93.86	98.62 / 94.75	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	97.71 / 92.56	97.75 / 92.86	96.87 / 90.90	97.76 / 92.06	97.18 / 91.61
da → en	94.88 / 92.30	95.57 / 92.79	96.61 / 93.29	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	95.43 / 92.58	97.33 / 93.74	96.52 / 93.28	97.06 / 93.60	97.25 / 93.71
en → de	98.00 / 85.26	98.66 / 87.16	98.94 / 88.25	97.88 / 85.32	98.37 / 86.99	98.73 / 87.42	97.98 / 84.32	98.69 / 87.49	97.80 / 84.32	98.55 / 86.96	98.76 / 87.99	98.45 / 86.75	98.28 / 87.17	97.75 / 84.73	98.51 / 87.15	98.58 / 87.22
de → en	95.43 / 89.92	95.98 / 90.71	96.62 / 91.25	96.86 / 91.50	97.06 / 91.63	96.18 / 90.43	95.80 / 88.68	95.96 / 90.35	95.79 / 90.41	96.88 / 91.45	96.77 / 91.30	95.64 / 90.64	97.01 / 91.68	96.28 / 91.04	96.78 / 91.42	96.93 / 91.56
en → el	92.77 / 86.77	96.12 / 90.66	96.90 / 92.41	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
el → en	89.96 / 88.16	92.20 / 89.38	93.60 / 90.10	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → es	97.52 / 91.08	97.79 / 92.01	98.11 / 92.32	97.05 / 89.59	97.57 / 90.63	98.07 / 91.88	97.76 / 90.38	97.98 / 91.90	97.29 / 89.69	97.65 / 91.16	98.01 / 91.77	97.15 / 89.41	97.32 / 89.73	96.89 / 89.03	97.83 / 91.07	97.73 / 91.07
es → en	94.11 / 90.21	95.51 / 90.83	96.22 / 91.32	96.03 / 91.20	96.41 / 91.42	95.42 / 90.22	94.92 / 88.25	95.53 / 90.40	95.43 / 90.46	96.06 / 91.14	96.33 / 91.28	91.35 / 88.18	96.61 / 91.48	95.89 / 90.85	96.08 / 91.36	96.17 / 91.39
en → fa	91.29 / 87.76	95.76 / 91.96	96.65 / 92.71	92.03 / 88.45	94.86 / 91.37	94.11 / 89.93	91.53 / 86.43	94.40 / 90.03	91.44 / 86.99	95.36 / 91.49	95.85 / 92.22	- / -	- / -	- / -	- / -	- / -
fa → en	87.32 / 84.38	90.78 / 85.68	93.11 / 87.02	92.94 / 87.32	93.73 / 87.82	90.49 / 85.05	87.55 / 82.54	90.97 / 85.42	88.93 / 84.65	92.47 / 87.07	93.13 / 87.37	- / -	- / -	- / -	- / -	- / -
en → fi	92.11 / 90.21	95.94 / 94.89	97.17 / 96.23	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	95.31 / 93.49	96.20 / 94.52	92.76 / 90.22	96.39 / 94.64	94.84 / 92.68
fi → en	89.48 / 89.20	92.60 / 90.42	94.23 / 90.84	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	92.15 / 90.00	94.72 / 90.81	92.35 / 89.89	94.43 / 90.99	94.60 / 90.97
en → fr	95.72 / 91.57	96.89 / 92.93	96.97 / 93.05	95.70 / 90.38	96.94 / 91.81	96.80 / 92.47	95.45 / 89.92	96.97 / 92.68	95.33 / 89.48	96.98 / 91.92	97.10 / 92.25	96.62 / 91.11	96.77 / 91.17	95.25 / 89.22	96.76 / 91.71	97.02 / 92.02
fr → en	93.51 / 91.37	94.46 / 92.00	95.20 / 92.39	96.10 / 92.46	96.20 / 92.73	95.03 / 91.50	94.08 / 89.63	94.91 / 91.56	95.35 / 91.56	96.05 / 92.45	95.85 / 92.25	92.46 / 90.56	96.21 / 92.70	95.78 / 92.29	95.90 / 92.54	95.89 / 92.64
en → he	87.79 / 79.92	94.14 / 87.65	95.68 / 89.66	91.47 / 84.40	94.21 / 87.82	90.18 / 81.22	86.99 / 77.13	90.45 / 81.86	88.62 / 79.61	94.10 / 87.21	95.13 / 88.30	- / -	- / -	- / -	- / -	- / -
he → en	89.92 / 85.71	93.24 / 87.51	94.62 / 88.50	94.37 / 88.29	95.24 / 89.01	91.09 / 85.77	89.52 / 83.63	91.59 / 86.24	89.79 / 85.52	94.61 / 88.29	94.90 / 88.62	- / -	- / -	- / -	- / -	- / -
en → hi	89.14 / 75.25	92.66 / 78.04	93.71 / 79.11	86.70 / 76.38	89.61 / 78.38	94.14 / 76.46	91.81 / 72.28	94.27 / 77.82	89.95 / 75.15	93.96 / 79.22	94.32 / 79.70	- / -	- / -	83.44 / 74.54	86.97 / 77.13	86.65 / 76.52
hi → en	90.52 / 77.12	93.06 / 78.38	94.17 / 80.09	93.71 / 81.86	94.89 / 82.57	92.25 / 77.84	91.51 / 76.44	93.05 / 78.55	91.70 / 79.79	94.13 / 81.61	95.02 / 81.51	- / -	- / -	92.56 / 80.92	94.73 / 82.03	94.60 / 81.87

Table 9: Reference-free English-centric evaluation results (XCOMET / COMETKiwi) of baseline models (Part II) and MiLMMT models on the FLORES+ benchmark.
Direction	Google Translate	NLLB-54.5B	Gemini 2.5 Pro	Gemini 3 Pro	GPT-5	MiLMMT-46-1B-v0.1	MiLMMT-46-1B-v1.0	MiLMMT-46-4B-v0.1	MiLMMT-46-4B-v1.0	MiLMMT-46-12B-v0.1	MiLMMT-46-12B-v1.0
en → ar	95.05 / 89.48	91.61 / 84.52	95.64 / 89.85	95.68 / 90.03	94.83 / 89.87	89.03 / 82.35	91.63 / 85.10	93.67 / 87.88	95.49 / 89.82	94.77 / 88.90	96.96 / 91.53
ar → en	92.15 / 87.00	90.63 / 85.83	92.75 / 87.02	92.56 / 87.07	92.00 / 86.93	90.24 / 85.76	91.34 / 86.04	92.39 / 86.93	92.66 / 87.06	93.11 / 87.28	93.35 / 87.37
en → az	88.43 / 86.98	85.93 / 82.51	93.82 / 92.04	94.15 / 92.24	94.28 / 92.14	84.73 / 83.88	88.48 / 87.01	92.15 / 90.09	94.86 / 92.54	93.63 / 91.42	96.67 / 93.96
az → en	85.50 / 88.87	81.21 / 87.65	86.56 / 88.83	86.65 / 88.77	86.08 / 88.74	80.83 / 86.51	83.97 / 86.94	86.01 / 88.53	87.50 / 88.38	86.60 / 88.84	88.19 / 88.80
en → bg	95.99 / 91.09	93.16 / 86.35	96.79 / 93.02	97.26 / 93.28	96.10 / 93.18	92.30 / 86.17	94.08 / 88.79	96.02 / 91.61	97.41 / 94.00	96.73 / 92.41	98.28 / 95.08
bg → en	95.26 / 91.09	92.30 / 89.50	95.36 / 91.09	95.47 / 91.16	94.87 / 91.10	94.17 / 90.26	94.80 / 90.39	95.64 / 91.17	96.00 / 91.33	95.70 / 91.37	95.89 / 91.39
en → bn	90.40 / 84.17	87.81 / 80.98	92.82 / 85.51	94.31 / 86.87	91.56 / 85.61	86.95 / 81.05	91.83 / 83.94	92.52 / 85.61	95.51 / 87.45	93.64 / 86.36	96.69 / 88.49
bn → en	90.65 / 86.72	87.64 / 85.56	90.92 / 86.64	91.11 / 86.85	90.77 / 86.65	86.46 / 84.91	88.56 / 85.21	89.81 / 86.39	91.06 / 86.30	90.99 / 86.81	91.94 / 86.78
en → ca	95.14 / 88.88	93.18 / 84.47	95.67 / 89.74	95.47 / 89.40	95.31 / 90.48	94.28 / 86.68	95.81 / 89.22	95.70 / 89.39	97.34 / 92.20	96.18 / 90.03	97.96 / 93.42
ca → en	95.20 / 92.37	92.78 / 90.99	95.10 / 92.60	95.12 / 92.59	94.63 / 92.60	94.75 / 91.99	94.57 / 91.94	95.18 / 92.65	95.42 / 92.74	95.66 / 92.94	95.74 / 92.91
en → cs	95.45 / 90.95	91.45 / 86.98	96.22 / 92.36	96.54 / 92.77	96.48 / 93.05	90.61 / 83.96	93.09 / 88.28	95.59 / 91.06	96.78 / 93.37	96.32 / 92.32	97.50 / 95.13
cs → en	95.59 / 92.14	93.20 / 90.60	95.74 / 91.89	95.74 / 91.94	95.17 / 92.16	94.28 / 90.88	95.05 / 91.19	95.76 / 91.98	96.20 / 92.24	96.04 / 92.32	96.41 / 92.49
en → da	97.66 / 92.49	96.26 / 89.10	98.34 / 93.57	98.37 / 93.80	98.10 / 94.01	96.12 / 89.10	97.01 / 90.72	97.77 / 92.65	98.58 / 94.40	98.22 / 93.60	98.96 / 95.25
da → en	97.01 / 93.73	94.70 / 91.94	97.03 / 93.56	97.23 / 93.65	96.67 / 93.52	96.48 / 93.08	96.75 / 93.30	97.36 / 93.79	97.63 / 93.82	97.36 / 93.81	97.56 / 93.89
en → de	98.77 / 87.88	97.64 / 83.49	98.49 / 87.48	98.66 / 87.91	98.01 / 87.27	96.93 / 81.77	97.57 / 83.60	98.41 / 86.63	98.75 / 87.57	98.58 / 87.35	99.10 / 88.99
de → en	96.79 / 91.41	95.46 / 90.17	96.68 / 91.25	96.79 / 91.27	96.38 / 91.28	96.04 / 90.58	96.29 / 90.69	96.71 / 91.36	97.17 / 91.69	96.96 / 91.55	97.34 / 91.76
en → el	95.24 / 89.95	93.03 / 84.24	96.28 / 91.31	96.19 / 91.44	95.57 / 91.44	90.34 / 84.29	92.22 / 86.04	95.14 / 90.09	96.72 / 91.98	95.85 / 91.24	97.59 / 93.15
el → en	94.42 / 90.65	93.10 / 89.57	94.26 / 90.54	94.41 / 90.54	93.89 / 90.53	92.61 / 89.44	93.69 / 89.70	94.60 / 90.62	95.08 / 90.68	94.77 / 90.81	95.10 / 90.95
en → es	97.30 / 90.46	95.88 / 86.63	97.50 / 90.48	97.34 / 90.38	97.27 / 91.13	96.42 / 88.53	97.43 / 90.27	97.46 / 90.42	98.42 / 92.56	97.81 / 91.13	98.81 / 93.48
es → en	95.88 / 91.16	92.47 / 87.66	95.91 / 91.17	96.10 / 91.10	95.67 / 91.20	95.67 / 90.75	96.10 / 91.04	96.38 / 91.36	96.59 / 91.45	96.56 / 91.50	96.84 / 91.72
en → fa	95.04 / 91.46	90.52 / 86.06	95.69 / 92.10	95.58 / 92.10	94.41 / 92.10	85.79 / 82.45	89.65 / 85.86	93.67 / 89.97	95.35 / 91.47	94.92 / 91.51	96.92 / 93.32
fa → en	93.83 / 87.95	90.75 / 86.40	94.07 / 87.76	93.84 / 87.73	94.18 / 87.79	90.17 / 85.89	91.84 / 86.51	93.27 / 87.46	93.91 / 87.66	94.09 / 87.99	94.57 / 88.11
en → fi	96.28 / 94.71	92.46 / 90.76	97.35 / 96.02	97.46 / 96.28	96.89 / 96.40	87.34 / 84.07	89.71 / 86.44	95.38 / 93.47	96.83 / 95.40	96.62 / 95.04	97.96 / 96.74
fi → en	94.67 / 91.00	91.37 / 89.66	94.88 / 90.81	94.82 / 90.85	94.25 / 90.76	92.16 / 89.47	93.13 / 89.59	94.96 / 90.92	95.27 / 91.02	95.54 / 91.18	95.86 / 91.16
en → fr	96.50 / 91.46	93.79 / 86.71	96.43 / 91.42	96.63 / 91.64	96.34 / 91.79	94.53 / 88.13	95.19 / 89.85	96.49 / 91.13	97.33 / 93.22	97.03 / 92.00	98.06 / 94.11
fr → en	95.46 / 92.25	94.34 / 91.25	95.46 / 92.23	95.73 / 92.05	95.31 / 92.32	95.80 / 92.24	96.16 / 92.52	96.43 / 92.74	96.59 / 92.90	96.31 / 92.72	96.60 / 92.93
en → he	93.68 / 86.87	91.81 / 83.62	94.60 / 88.54	94.90 / 88.83	93.51 / 89.08	86.04 / 77.77	87.35 / 78.58	93.32 / 86.35	95.09 / 88.32	94.69 / 88.43	96.70 / 90.58
he → en	95.08 / 88.98	91.81 / 86.96	95.22 / 88.83	95.35 / 88.89	94.21 / 88.77	91.86 / 87.09	92.52 / 87.28	95.17 / 88.76	95.09 / 88.86	95.58 / 89.11	95.53 / 89.22
en → hi	92.01 / 79.33	83.66 / 75.63	91.30 / 78.96	92.64 / 79.50	91.93 / 78.18	82.47 / 73.39	88.77 / 75.10	89.00 / 77.77	93.61 / 78.40	91.04 / 79.36	95.25 / 80.00
hi → en	94.53 / 81.74	94.04 / 81.55	95.14 / 82.28	95.14 / 82.72	94.88 / 82.06	92.11 / 81.16	92.60 / 81.31	94.61 / 82.38	95.16 / 82.28	95.01 / 82.61	95.41 / 82.24

Table 10: Reference-free Chinese-centric evaluation results (XCOMET / COMETKiwi) of baseline models (Part I) on the FLORES+ benchmark.
Direction	TranslateGemma-4B	TranslateGemma-12B	TranslateGemma-27B	GemmaX2-28-2B	GemmaX2-28-9B	Hunyuan-MT-7B	HY-MT1.5-1.8B	HY-MT1.5-7B	HY-MT2-1.8B	HY-MT2-7B	HY-MT2-30B-A3B	Seed-X-Instruct-7B	Seed-X-PPO-7B	Tower-Plus-2B	Tower-Plus-9B	Tower-Plus-72B
zhs → ar	78.48 / 73.15	81.62 / 76.79	83.13 / 77.26	77.98 / 71.25	82.40 / 74.60	83.67 / 76.35	80.80 / 73.82	84.28 / 77.25	81.46 / 73.69	83.97 / 76.58	83.39 / 75.80	78.17 / 71.29	82.11 / 75.08	- / -	- / -	- / -
ar → zhs	68.08 / 68.66	70.55 / 70.69	71.60 / 71.19	68.83 / 70.57	71.75 / 72.40	73.04 / 71.40	70.80 / 69.76	73.87 / 71.00	72.49 / 70.83	74.49 / 71.46	74.36 / 72.82	65.91 / 70.45	65.42 / 69.60	- / -	- / -	- / -
zhs → az	40.12 / 51.10	75.66 / 81.55	81.98 / 84.82	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
az → zhs	42.27 / 73.69	49.12 / 78.89	50.71 / 79.92	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
zhs → bg	88.55 / 81.80	92.12 / 85.72	92.94 / 86.44	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
bg → zhs	66.45 / 73.69	68.36 / 76.21	69.48 / 76.60	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
zhs → bn	84.89 / 72.44	87.52 / 76.05	89.18 / 76.30	84.72 / 73.63	88.10 / 75.71	87.97 / 74.38	84.46 / 71.06	89.20 / 77.23	85.64 / 72.10	90.41 / 75.98	90.58 / 76.10	- / -	- / -	- / -	- / -	- / -
bn → zhs	58.79 / 70.51	62.46 / 73.46	63.78 / 74.14	61.59 / 74.57	64.42 / 76.75	61.73 / 70.15	63.92 / 70.39	64.77 / 70.57	64.98 / 71.01	67.08 / 72.70	68.69 / 76.20	- / -	- / -	- / -	- / -	- / -
zhs → ca	82.24 / 76.08	86.99 / 80.71	88.26 / 81.97	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
ca → zhs	63.09 / 75.26	68.51 / 79.12	67.36 / 79.76	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
zhs → cs	75.97 / 83.84	80.60 / 88.03	83.05 / 88.77	80.85 / 84.58	85.01 / 87.48	84.27 / 88.01	78.02 / 82.47	84.40 / 88.18	81.81 / 83.99	84.78 / 87.61	84.51 / 87.83	76.31 / 84.12	83.44 / 87.93	79.58 / 83.64	83.94 / 87.73	84.49 / 87.40
cs → zhs	53.03 / 80.59	57.01 / 83.56	56.60 / 83.75	54.51 / 81.37	57.24 / 82.88	60.67 / 84.48	58.41 / 81.90	61.01 / 83.50	58.88 / 81.51	61.98 / 84.18	59.96 / 84.62	48.66 / 81.84	49.70 / 82.70	51.80 / 79.16	55.00 / 82.37	55.30 / 82.89
zhs → da	90.51 / 83.73	93.00 / 86.57	93.80 / 87.71	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	90.59 / 83.59	93.23 / 86.57	91.29 / 83.50	93.36 / 85.69	93.15 / 85.29
da → zhs	75.25 / 81.93	78.48 / 84.18	78.97 / 84.75	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	72.49 / 82.93	73.27 / 83.63	74.91 / 80.68	77.84 / 83.09	77.73 / 83.94
zhs → de	93.58 / 78.67	95.69 / 81.38	96.15 / 82.46	94.44 / 79.86	95.70 / 82.26	96.15 / 81.75	93.99 / 76.73	96.17 / 81.76	94.87 / 79.66	96.30 / 82.12	96.30 / 82.95	94.33 / 80.99	95.99 / 82.56	93.85 / 78.31	95.80 / 82.39	95.91 / 82.30
de → zhs	78.15 / 82.57	82.16 / 85.62	81.83 / 86.05	81.13 / 82.21	82.44 / 83.17	83.74 / 87.67	81.20 / 85.52	83.07 / 87.28	82.34 / 84.14	83.48 / 86.34	83.39 / 86.45	77.35 / 82.49	76.07 / 84.21	78.45 / 80.33	81.78 / 83.29	81.66 / 83.53
zhs → el	80.70 / 71.70	86.61 / 76.82	87.38 / 77.67	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
el → zhs	57.34 / 62.73	60.61 / 65.49	61.18 / 65.91	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
zhs → en	96.37 / 86.48	97.02 / 87.03	97.34 / 87.76	97.01 / 87.46	97.18 / 87.85	97.26 / 87.51	96.43 / 85.71	97.15 / 87.42	96.73 / 87.09	97.46 / 87.64	97.62 / 87.92	96.63 / 87.03	97.07 / 87.65	96.54 / 87.16	97.08 / 87.69	97.28 / 87.79
en → zhs	91.98 / 87.14	94.05 / 88.92	94.45 / 89.86	92.50 / 87.66	93.21 / 88.48	95.56 / 89.93	93.68 / 88.42	95.04 / 89.84	93.08 / 88.03	95.06 / 89.91	95.11 / 90.01	93.41 / 88.63	94.31 / 89.93	90.60 / 85.96	92.33 / 88.05	93.36 / 88.95
zhs → es	89.13 / 79.18	90.31 / 80.42	90.89 / 80.66	90.48 / 79.64	91.57 / 80.95	91.55 / 79.99	90.74 / 78.09	91.82 / 80.06	91.14 / 78.80	92.80 / 80.55	92.28 / 80.88	89.53 / 80.04	91.84 / 81.10	89.53 / 78.45	91.42 / 80.80	91.70 / 81.48
es → zhs	82.86 / 79.89	85.39 / 82.74	85.74 / 83.47	83.12 / 78.17	84.94 / 80.11	87.32 / 84.86	85.88 / 82.11	87.53 / 85.02	85.75 / 80.53	87.58 / 83.75	87.28 / 83.13	79.39 / 77.85	80.55 / 81.20	80.84 / 76.66	83.31 / 78.69	84.50 / 79.71
zhs → fa	83.93 / 78.59	88.99 / 82.95	89.47 / 83.78	85.47 / 79.52	88.28 / 81.88	86.32 / 80.00	82.69 / 77.31	88.00 / 82.03	84.82 / 78.52	89.49 / 82.72	89.37 / 82.61	- / -	- / -	- / -	- / -	- / -
fa → zhs	73.15 / 73.14	77.01 / 75.97	78.52 / 77.11	75.31 / 74.34	77.38 / 76.38	78.11 / 76.38	75.41 / 73.26	79.96 / 76.59	76.82 / 73.74	81.05 / 77.23	80.69 / 77.84	- / -	- / -	- / -	- / -	- / -
zhs → fi	78.25 / 84.10	86.43 / 89.55	87.94 / 91.28	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	81.13 / 85.82	86.27 / 88.95	81.84 / 84.66	87.10 / 89.19	85.37 / 87.68
fi → zhs	51.78 / 81.33	58.78 / 83.94	58.76 / 84.36	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	48.74 / 83.40	50.42 / 83.44	53.33 / 80.88	54.77 / 82.73	57.60 / 84.19
zhs → fr	77.61 / 78.80	79.38 / 81.08	80.52 / 81.24	81.78 / 80.76	83.88 / 82.45	81.91 / 81.35	80.34 / 78.52	82.34 / 81.06	82.18 / 80.26	83.52 / 82.60	83.58 / 82.37	79.28 / 81.08	82.39 / 81.99	79.16 / 79.26	82.68 / 81.99	83.27 / 82.58
fr → zhs	66.95 / 77.46	69.83 / 80.86	70.73 / 82.41	65.94 / 74.34	67.58 / 76.05	74.52 / 83.88	71.65 / 80.80	75.00 / 84.23	69.36 / 77.94	73.90 / 82.16	72.38 / 80.62	61.52 / 75.43	63.74 / 79.59	62.97 / 72.48	66.90 / 76.02	67.00 / 76.75
zhs → he	74.10 / 69.04	83.12 / 77.32	84.16 / 78.19	79.53 / 73.07	83.94 / 77.15	76.67 / 71.23	80.78 / 63.21	78.94 / 73.19	78.62 / 70.90	85.83 / 77.89	84.86 / 77.41	- / -	- / -	- / -	- / -	- / -
he → zhs	70.93 / 71.50	75.31 / 74.22	76.38 / 75.01	74.17 / 73.60	75.71 / 74.82	75.40 / 74.20	74.15 / 71.48	76.22 / 73.25	75.86 / 72.04	79.27 / 74.88	79.87 / 76.09	- / -	- / -	- / -	- / -	- / -

Table 11: Reference-free Chinese-centric evaluation results (XCOMET / COMETKiwi) of baseline models (Part II) and MiLMMT models on the FLORES+ benchmark.
Direction	Google Translate	NLLB-54.5B	Gemini 2.5 Pro	Gemini 3 Pro	GPT-5	MiLMMT-46-1B-v0.1	MiLMMT-46-1B-v1.0	MiLMMT-46-4B-v0.1	MiLMMT-46-4B-v1.0	MiLMMT-46-12B-v0.1	MiLMMT-46-12B-v1.0
zhs → ar	79.53 / 73.09	79.30 / 71.84	82.51 / 74.67	83.14 / 74.76	83.06 / 75.46	74.88 / 68.37	77.85 / 71.85	81.05 / 73.92	83.96 / 76.94	83.03 / 75.40	84.86 / 78.05
ar → zhs	66.40 / 70.69	52.06 / 58.70	70.50 / 72.42	71.81 / 72.89	71.76 / 73.05	67.78 / 68.55	71.52 / 70.30	70.30 / 71.67	73.99 / 72.37	71.87 / 73.29	75.00 / 73.59
zhs → az	76.37 / 79.72	74.45 / 76.04	84.65 / 85.37	84.75 / 85.69	85.47 / 86.09	72.42 / 77.31	76.26 / 80.69	82.79 / 84.23	84.44 / 86.17	84.49 / 85.74	86.96 / 87.57
az → zhs	42.77 / 79.66	35.20 / 69.50	51.11 / 81.48	51.15 / 82.01	53.65 / 82.06	44.88 / 75.64	50.64 / 77.61	49.20 / 80.64	54.71 / 80.90	50.47 / 81.51	57.09 / 81.96
zhs → bg	90.64 / 82.96	87.00 / 78.52	92.39 / 85.37	92.66 / 85.35	93.18 / 86.06	86.60 / 78.50	88.93 / 81.56	92.11 / 84.75	93.20 / 86.45	92.86 / 85.62	93.85 / 87.51
bg → zhs	63.15 / 73.91	41.99 / 52.23	67.76 / 76.00	67.38 / 75.83	68.75 / 76.29	64.10 / 72.59	69.84 / 74.82	67.40 / 75.25	72.61 / 77.68	68.25 / 76.39	73.58 / 78.85
zhs → bn	83.30 / 70.64	80.46 / 67.99	87.97 / 75.47	88.97 / 76.23	89.13 / 76.45	79.75 / 69.11	85.07 / 74.04	87.50 / 75.32	90.12 / 78.07	88.52 / 76.34	91.34 / 78.99
bn → zhs	59.53 / 74.29	44.52 / 62.58	63.45 / 76.97	64.11 / 77.01	66.13 / 76.99	57.77 / 71.90	63.58 / 73.08	63.27 / 75.41	66.63 / 75.81	64.41 / 77.06	68.68 / 76.81
zhs → ca	84.07 / 78.03	80.90 / 74.83	86.85 / 80.32	87.58 / 80.52	88.66 / 81.41	83.82 / 76.11	86.97 / 79.47	88.01 / 80.52	89.23 / 82.30	88.46 / 81.14	90.52 / 83.46
ca → zhs	63.44 / 76.00	48.24 / 58.39	68.24 / 79.11	68.36 / 77.90	70.48 / 78.63	64.11 / 73.42	68.62 / 77.47	67.46 / 76.68	72.75 / 81.12	68.94 / 78.57	73.97 / 82.41
zhs → cs	81.01 / 85.66	75.05 / 80.70	84.32 / 87.73	85.25 / 88.25	85.86 / 88.66	75.07 / 79.51	78.98 / 83.85	84.87 / 87.06	84.94 / 88.82	85.68 / 88.18	85.49 / 89.68
cs → zhs	51.19 / 81.22	32.25 / 57.69	56.04 / 83.44	56.43 / 83.36	59.19 / 83.78	52.62 / 78.43	57.87 / 80.90	56.18 / 82.45	60.88 / 84.63	57.03 / 83.37	62.83 / 85.64
zhs → da	91.60 / 84.99	89.12 / 81.43	93.79 / 86.68	94.30 / 87.05	94.69 / 87.76	88.95 / 80.60	91.01 / 82.86	93.30 / 85.80	93.83 / 87.49	94.23 / 86.85	95.46 / 88.70
da → zhs	75.73 / 83.06	45.99 / 54.52	78.97 / 84.62	78.49 / 84.10	81.13 / 84.96	75.43 / 80.93	79.84 / 83.24	77.97 / 83.48	81.72 / 85.70	79.62 / 84.27	83.63 / 86.74
zhs → de	95.06 / 81.56	91.19 / 77.99	95.98 / 83.00	95.97 / 83.11	96.05 / 83.15	92.12 / 75.59	93.72 / 77.56	95.45 / 81.06	96.57 / 82.38	96.10 / 82.89	96.75 / 83.29
de → zhs	79.17 / 81.82	58.12 / 58.51	82.37 / 84.28	82.13 / 83.64	82.90 / 84.99	78.96 / 80.46	82.05 / 84.40	81.41 / 83.08	84.12 / 86.96	82.88 / 84.12	84.90 / 88.15
zhs → el	85.76 / 74.42	83.40 / 71.31	88.12 / 76.97	88.82 / 77.25	88.82 / 77.81	79.06 / 70.31	81.37 / 72.73	87.32 / 76.83	88.31 / 78.33	88.37 / 77.60	89.58 / 79.20
el → zhs	54.66 / 63.81	41.15 / 52.10	59.02 / 66.32	58.01 / 65.43	60.91 / 66.06	56.00 / 61.42	62.05 / 64.67	58.78 / 65.55	64.34 / 67.64	59.90 / 66.49	66.50 / 69.22
zhs → en	97.17 / 87.37	95.65 / 85.36	97.02 / 87.53	97.04 / 87.64	97.14 / 87.75	96.36 / 86.85	96.48 / 87.21	96.97 / 87.61	97.47 / 87.95	97.18 / 87.90	97.61 / 88.27
en → zhs	92.46 / 88.04	74.32 / 62.17	93.33 / 88.78	93.43 / 88.91	93.40 / 88.80	90.42 / 85.65	92.61 / 87.55	93.05 / 88.09	94.99 / 89.67	93.89 / 88.80	95.79 / 90.58
zhs → es	90.02 / 80.34	87.51 / 77.80	91.15 / 81.03	91.55 / 81.25	91.25 / 80.89	89.29 / 78.24	90.96 / 79.14	91.70 / 80.69	92.56 / 80.89	92.00 / 81.26	92.95 / 81.66
es → zhs	81.29 / 77.19	53.43 / 47.01	85.04 / 80.28	85.40 / 79.66	85.98 / 80.98	81.61 / 75.98	85.44 / 81.65	83.87 / 79.51	87.32 / 84.32	85.02 / 80.44	88.46 / 85.16
zhs → fa	86.07 / 79.52	82.13 / 76.25	89.03 / 81.86	88.99 / 82.00	89.02 / 82.51	78.17 / 73.43	82.89 / 77.46	87.22 / 80.76	89.18 / 83.11	88.66 / 81.86	91.37 / 85.01
fa → zhs	71.89 / 73.85	55.41 / 60.70	76.80 / 76.63	76.59 / 76.22	77.87 / 77.22	71.11 / 71.31	75.95 / 74.34	76.36 / 75.17	79.91 / 77.65	77.63 / 77.05	81.76 / 79.17
zhs → fi	84.70 / 88.28	77.46 / 81.62	89.25 / 90.52	89.73 / 90.73	89.97 / 91.13	74.15 / 77.15	78.83 / 80.99	87.12 / 88.11	88.30 / 90.20	88.72 / 90.02	90.98 / 91.90
fi → zhs	52.32 / 83.01	32.23 / 63.73	58.05 / 84.62	58.37 / 84.76	60.52 / 84.73	52.92 / 79.30	58.08 / 80.54	58.01 / 83.80	60.81 / 84.57	57.99 / 84.81	64.79 / 85.97
zhs → fr	80.48 / 81.63	76.83 / 77.16	81.98 / 82.61	82.96 / 82.66	82.35 / 82.25	78.94 / 77.20	80.93 / 79.08	83.04 / 82.05	83.19 / 81.94	83.72 / 82.68	83.82 / 82.49
fr → zhs	62.20 / 72.88	40.52 / 46.44	67.53 / 77.24	66.92 / 75.77	70.02 / 76.72	64.44 / 71.62	71.16 / 78.29	67.30 / 75.61	74.32 / 82.51	68.68 / 76.60	76.01 / 84.48
zhs → he	79.86 / 73.53	78.83 / 71.00	83.63 / 76.94	83.95 / 77.51	84.89 / 77.59	71.88 / 66.49	74.79 / 70.49	83.00 / 75.87	85.22 / 78.47	84.78 / 77.38	87.13 / 79.92
he → zhs	72.41 / 73.67	48.52 / 54.19	75.85 / 75.77	76.04 / 75.84	76.96 / 76.06	71.41 / 70.67	75.36 / 72.69	74.57 / 74.52	78.17 / 75.84	75.54 / 75.44	78.91 / 76.80

Table 12: Reference-based English-centric evaluation results (spBLEU / XCOMET) of baseline models (Part I) on the FLORES+ benchmark.
Direction	TranslateGemma-4B	TranslateGemma-12B	TranslateGemma-27B	GemmaX2-28-2B	GemmaX2-28-9B	Hunyuan-MT-7B	HY-MT1.5-1.8B	HY-MT1.5-7B	HY-MT2-1.8B	HY-MT2-7B	HY-MT2-30B-A3B	Seed-X-Instruct-7B	Seed-X-PPO-7B	Tower-Plus-2B	Tower-Plus-9B	Tower-Plus-72B
en → ar	31.42 / 92.06	33.73 / 94.89	33.79 / 96.20	37.71 / 90.33	41.91 / 93.56	32.27 / 95.46	27.96 / 92.30	32.68 / 95.57	34.00 / 92.05	38.39 / 94.93	38.80 / 95.37	41.78 / 92.87	42.62 / 93.38	- / -	- / -	- / -
ar → en	34.36 / 89.18	36.37 / 92.60	40.06 / 94.09	45.26 / 92.01	48.05 / 94.18	32.52 / 92.22	26.51 / 89.55	33.42 / 92.72	34.33 / 89.79	42.98 / 93.51	44.06 / 94.45	47.05 / 92.38	43.72 / 94.36	- / -	- / -	- / -
en → az	6.82 / 39.69	18.27 / 78.24	21.93 / 86.27	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
az → en	20.84 / 76.12	23.68 / 85.74	26.14 / 88.98	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → bn	24.35 / 87.37	24.81 / 91.23	26.10 / 92.14	31.60 / 88.23	33.13 / 90.80	24.89 / 92.34	22.67 / 88.91	25.77 / 92.72	25.82 / 88.82	28.90 / 92.54	27.63 / 93.28	- / -	- / -	- / -	- / -	- / -
bn → en	28.66 / 85.75	31.33 / 89.90	34.53 / 91.73	39.77 / 89.28	42.80 / 91.36	25.16 / 85.21	21.36 / 85.78	28.52 / 86.98	27.55 / 85.04	35.89 / 89.31	37.36 / 91.97	- / -	- / -	- / -	- / -	- / -
en → bg	37.72 / 92.11	39.33 / 96.08	40.82 / 97.21	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
bg → en	37.76 / 92.29	39.51 / 94.25	42.45 / 95.71	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → ca	38.42 / 91.34	42.09 / 95.57	39.11 / 96.30	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
ca → en	42.43 / 92.31	44.23 / 94.61	46.79 / 95.66	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → cs	31.32 / 89.73	34.86 / 94.60	34.17 / 95.36	40.65 / 91.87	43.74 / 94.23	38.43 / 94.77	33.69 / 90.20	38.16 / 94.70	38.83 / 90.52	43.00 / 94.87	42.99 / 95.54	40.76 / 93.64	44.51 / 94.48	39.95 / 91.66	43.28 / 94.33	42.19 / 93.64
cs → en	37.57 / 93.31	38.32 / 95.06	41.36 / 95.92	46.16 / 95.28	47.78 / 96.08	35.53 / 94.64	30.46 / 93.45	37.11 / 94.52	38.77 / 93.47	44.18 / 95.75	44.35 / 96.26	44.89 / 93.35	45.32 / 96.13	44.70 / 94.73	47.56 / 95.90	47.41 / 96.17
en → zhs	31.02 / 91.99	31.96 / 94.45	33.60 / 95.15	37.94 / 92.88	39.85 / 93.72	31.03 / 96.04	30.57 / 93.81	29.94 / 95.58	35.79 / 93.47	36.01 / 95.64	37.68 / 95.71	37.60 / 94.01	34.09 / 95.28	36.39 / 90.84	39.92 / 93.32	42.21 / 94.51
zhs → en	27.24 / 95.63	28.90 / 96.71	30.94 / 97.27	34.68 / 96.48	36.57 / 96.96	29.57 / 96.86	23.89 / 96.00	27.48 / 96.82	26.95 / 96.17	28.98 / 97.04	29.82 / 97.68	36.47 / 96.46	33.19 / 96.86	32.02 / 95.53	34.78 / 96.77	36.72 / 97.26
en → zht	12.41 / 89.41	12.04 / 92.73	27.31 / 94.66	- / -	- / -	8.43 / 94.26	8.44 / 92.63	24.16 / 94.27	10.72 / 91.19	29.46 / 94.64	29.71 / 95.18	- / -	- / -	30.11 / 88.95	33.01 / 92.11	31.59 / 93.14
zht → en	25.80 / 93.78	27.46 / 95.70	29.78 / 96.49	- / -	- / -	27.86 / 95.48	22.01 / 93.96	27.98 / 95.33	27.07 / 93.67	32.18 / 95.77	33.27 / 96.62	- / -	- / -	29.92 / 93.99	33.39 / 96.06	35.84 / 96.62
en → da	42.35 / 95.94	43.94 / 97.72	41.86 / 98.22	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	51.01 / 97.19	54.91 / 97.38	50.91 / 96.01	53.66 / 97.16	51.88 / 96.65
da → en	45.10 / 95.11	46.14 / 96.18	47.73 / 97.13	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	54.25 / 95.61	53.33 / 97.22	53.28 / 96.27	55.60 / 97.26	54.95 / 97.25
en → de	39.21 / 97.73	40.28 / 98.50	41.85 / 98.89	45.58 / 97.66	49.34 / 98.26	40.68 / 98.60	35.01 / 97.66	41.18 / 98.58	40.89 / 97.48	46.03 / 98.42	46.78 / 98.72	49.02 / 98.30	49.72 / 97.95	44.95 / 97.50	48.62 / 98.38	48.11 / 98.43
de → en	41.12 / 95.49	42.10 / 96.49	44.22 / 97.05	49.53 / 96.72	50.85 / 97.12	39.01 / 96.47	33.40 / 95.93	40.03 / 96.30	42.34 / 95.61	47.24 / 97.01	47.59 / 97.39	50.92 / 96.28	49.19 / 97.13	48.38 / 96.22	50.24 / 97.18	51.11 / 97.33
en → el	30.38 / 88.80	34.32 / 93.37	35.38 / 95.49	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
el → en	33.40 / 90.76	35.27 / 93.40	37.86 / 94.73	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
en → tl	30.08 / 86.91	32.59 / 90.38	35.55 / 90.46	38.05 / 87.46	40.11 / 89.36	26.18 / 84.06	25.67 / 87.60	26.74 / 84.10	32.79 / 87.79	36.27 / 90.84	35.48 / 91.17	- / -	- / -	- / -	- / -	- / -
tl → en	39.74 / 91.04	40.59 / 93.52	45.31 / 94.87	52.76 / 93.48	55.75 / 94.68	35.20 / 92.49	30.05 / 91.40	37.00 / 92.61	40.61 / 90.82	48.18 / 93.76	48.79 / 94.59	- / -	- / -	- / -	- / -	- / -
en → fi	27.19 / 87.98	30.98 / 94.05	32.21 / 95.63	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	43.49 / 93.72	45.60 / 94.74	32.93 / 89.80	37.95 / 94.42	35.31 / 92.43
fi → en	33.37 / 90.81	35.72 / 94.06	37.14 / 95.34	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	41.75 / 93.24	39.90 / 95.10	38.80 / 92.84	42.19 / 95.34	42.34 / 95.55
en → fr	47.04 / 93.78	47.47 / 95.77	44.06 / 96.17	54.59 / 93.97	57.60 / 95.74	44.54 / 95.53	39.72 / 93.16	44.60 / 95.84	47.94 / 93.34	52.04 / 95.77	51.75 / 96.07	57.41 / 95.19	57.91 / 95.15	54.32 / 93.61	56.14 / 95.79	56.85 / 95.74
fr → en	42.48 / 94.60	42.79 / 95.35	45.11 / 96.11	50.27 / 96.18	51.34 / 96.54	38.67 / 95.53	33.00 / 94.62	39.79 / 95.42	43.00 / 95.27	47.24 / 96.28	48.15 / 96.54	49.71 / 93.94	49.71 / 96.62	49.41 / 96.00	50.49 / 96.47	51.76 / 96.52
en → he	29.39 / 84.93	34.42 / 93.30	37.17 / 95.24	42.71 / 89.97	47.20 / 93.46	29.60 / 88.09	25.78 / 84.36	30.48 / 88.73	34.87 / 86.64	42.37 / 93.34	42.47 / 94.57	- / -	- / -	- / -	- / -	- / -
he → en	36.88 / 90.28	40.06 / 93.98	43.73 / 95.79	48.36 / 94.32	50.92 / 95.79	34.08 / 91.60	27.92 / 89.12	35.90 / 92.11	36.36 / 89.04	45.64 / 94.56	46.72 / 95.64	- / -	- / -	- / -	- / -	- / -

Table 13: Reference-based English-centric evaluation results (spBLEU / XCOMET) of baseline models (Part II) and MiLMMT models on the FLORES+ benchmark.
Direction	Google Translate	NLLB-54.5B	Gemini 2.5 Pro	Gemini 3 Pro	GPT-5	MiLMMT-46-1B-v0.1	MiLMMT-46-1B-v1.0	MiLMMT-46-4B-v0.1	MiLMMT-46-4B-v1.0	MiLMMT-46-12B-v0.1	MiLMMT-46-12B-v1.0
en → ar	44.90 / 95.14	43.03 / 90.90	41.55 / 95.43	43.60 / 95.78	38.68 / 94.69	34.99 / 87.04	33.66 / 88.67	40.31 / 93.00	37.48 / 93.95	42.61 / 94.34	38.48 / 95.90
ar → en	50.99 / 94.87	48.26 / 92.50	49.55 / 95.54	49.26 / 95.67	46.74 / 95.10	42.43 / 90.13	41.25 / 90.60	46.94 / 93.51	45.54 / 93.63	48.92 / 94.78	47.23 / 94.96
en → az	26.51 / 82.61	24.63 / 79.16	27.41 / 90.16	28.07 / 90.47	25.85 / 89.46	22.27 / 76.34	21.73 / 79.44	25.64 / 86.38	24.96 / 88.19	26.95 / 88.76	25.89 / 90.61
az → en	33.24 / 90.76	31.94 / 87.01	31.77 / 91.71	32.12 / 91.82	29.73 / 91.43	26.22 / 84.35	25.31 / 86.04	29.18 / 89.82	28.23 / 90.22	30.49 / 90.89	29.24 / 91.71
en → bn	37.55 / 88.41	36.04 / 85.21	33.47 / 90.64	32.86 / 92.28	28.46 / 89.72	27.62 / 83.29	25.69 / 86.41	31.66 / 89.69	28.59 / 92.01	32.73 / 91.35	29.01 / 93.51
bn → en	44.97 / 91.90	42.22 / 89.21	44.35 / 92.95	44.18 / 93.28	41.50 / 93.03	36.85 / 86.34	36.20 / 87.79	41.63 / 90.72	40.39 / 91.37	43.04 / 91.92	41.73 / 92.52
en → bg	53.44 / 95.44	49.98 / 92.10	51.39 / 96.37	52.29 / 96.67	47.61 / 95.70	45.61 / 90.51	43.85 / 91.80	50.79 / 95.03	47.97 / 95.89	51.87 / 95.99	47.29 / 96.79
bg → en	51.32 / 95.77	47.01 / 93.54	49.37 / 95.97	50.52 / 96.21	47.86 / 95.65	45.53 / 93.52	44.19 / 94.01	48.59 / 95.45	47.04 / 95.80	49.75 / 95.87	47.88 / 96.05
en → ca	52.33 / 94.98	48.85 / 92.54	50.96 / 95.73	52.19 / 95.50	46.53 / 95.40	46.20 / 92.63	44.31 / 93.51	49.40 / 94.87	45.58 / 95.51	50.78 / 95.59	47.46 / 96.47
ca → en	55.13 / 95.67	52.58 / 93.77	53.02 / 96.02	54.53 / 96.33	52.30 / 96.08	49.90 / 94.05	48.86 / 94.42	52.34 / 95.61	51.10 / 95.89	53.84 / 96.18	52.40 / 96.27
en → cs	46.66 / 94.30	42.38 / 90.10	44.94 / 95.55	46.44 / 95.62	43.66 / 95.67	37.56 / 87.71	36.82 / 89.82	42.69 / 93.73	40.96 / 94.92	44.91 / 95.14	41.89 / 95.79
cs → en	49.97 / 96.15	45.31 / 94.15	47.78 / 96.35	48.71 / 96.56	46.31 / 96.11	44.22 / 94.02	42.85 / 94.59	46.94 / 95.94	45.17 / 96.22	48.39 / 96.37	46.67 / 96.58
en → zhs	44.05 / 93.84	26.61 / 76.72	39.42 / 94.26	40.26 / 94.43	37.73 / 94.19	35.03 / 90.32	34.22 / 91.77	38.03 / 93.21	36.84 / 94.55	38.98 / 94.11	37.33 / 95.28
zhs → en	39.59 / 97.18	36.13 / 95.04	38.26 / 97.28	38.12 / 97.38	34.82 / 97.52	32.44 / 95.14	30.84 / 95.57	35.40 / 96.59	33.74 / 96.79	36.60 / 97.21	34.94 / 97.38
en → zht	33.84 / 92.75	12.44 / 56.52	33.63 / 93.86	35.49 / 94.20	31.41 / 93.59	27.61 / 88.67	26.77 / 90.43	30.76 / 92.20	28.36 / 93.69	32.45 / 93.48	29.70 / 94.92
zht → en	37.95 / 96.15	32.27 / 92.00	37.02 / 96.88	37.13 / 97.20	33.52 / 96.86	30.77 / 93.61	29.61 / 94.05	33.21 / 95.81	31.98 / 95.87	34.77 / 96.42	33.43 / 96.71
en → da	55.61 / 97.25	50.04 / 95.66	53.61 / 98.02	54.62 / 98.18	51.23 / 97.82	49.11 / 95.10	47.78 / 95.63	52.57 / 97.14	50.55 / 97.60	54.70 / 97.81	51.72 / 98.31
da → en	58.03 / 97.39	53.07 / 95.43	54.84 / 97.40	55.88 / 97.52	53.48 / 97.23	53.32 / 95.92	52.19 / 96.14	55.01 / 97.23	53.94 / 97.43	55.72 / 97.45	54.44 / 97.49
en → de	49.24 / 98.81	46.62 / 97.50	49.54 / 98.55	51.94 / 98.67	47.64 / 98.12	43.10 / 96.45	40.74 / 96.93	47.32 / 98.28	45.88 / 98.45	48.91 / 98.50	46.04 / 98.83
de → en	52.41 / 97.24	49.80 / 96.06	50.31 / 97.16	51.05 / 97.31	49.52 / 97.12	47.82 / 95.87	46.73 / 95.99	49.91 / 96.84	48.51 / 97.05	51.28 / 97.20	49.43 / 97.33
en → el	42.95 / 93.61	38.73 / 91.03	42.78 / 95.01	43.62 / 94.86	41.10 / 94.26	35.76 / 87.18	35.47 / 88.87	40.61 / 93.29	39.40 / 94.34	42.02 / 94.31	41.05 / 95.52
el → en	45.33 / 94.85	43.70 / 93.79	44.00 / 95.12	43.51 / 95.45	41.68 / 95.03	38.94 / 91.92	37.33 / 92.47	42.19 / 94.64	40.84 / 94.97	43.52 / 95.33	42.16 / 95.46
en → tl	39.72 / 82.85	38.34 / 83.28	38.74 / 90.18	40.47 / 90.53	38.45 / 90.12	33.97 / 83.60	32.48 / 86.15	37.96 / 87.95	35.10 / 90.14	39.53 / 89.18	36.66 / 91.77
tl → en	58.10 / 94.74	54.58 / 92.86	56.49 / 95.01	56.90 / 95.08	52.61 / 95.00	49.06 / 91.45	47.64 / 91.94	53.41 / 94.21	51.93 / 94.57	55.41 / 94.92	53.80 / 95.10
en → fi	42.30 / 94.59	36.57 / 90.00	42.85 / 96.63	44.74 / 96.74	40.05 / 95.98	30.78 / 83.81	30.29 / 84.93	37.12 / 93.26	36.22 / 94.29	39.92 / 95.00	39.06 / 95.92
fi → en	44.26 / 95.41	41.22 / 92.87	42.56 / 95.72	43.23 / 96.07	41.05 / 95.78	37.82 / 92.00	37.09 / 92.64	41.12 / 95.09	40.01 / 95.13	42.35 / 95.80	40.91 / 95.80
en → fr	59.40 / 95.81	56.16 / 92.44	56.82 / 95.68	59.16 / 96.06	53.18 / 95.76	51.67 / 92.44	48.76 / 93.11	55.45 / 95.21	50.97 / 95.51	57.64 / 95.74	51.68 / 96.42
fr → en	54.39 / 96.33	51.54 / 95.11	51.46 / 96.39	52.26 / 96.77	49.82 / 96.45	48.37 / 95.58	47.24 / 95.80	50.40 / 96.60	48.67 / 96.58	51.60 / 96.71	50.23 / 96.90
en → he	51.65 / 93.63	46.82 / 90.79	48.01 / 94.89	51.50 / 95.15	45.42 / 94.02	37.22 / 83.68	35.53 / 84.96	45.45 / 92.33	41.66 / 93.60	48.52 / 94.52	43.61 / 95.36
he → en	53.64 / 95.84	49.02 / 92.85	52.37 / 96.63	52.61 / 96.92	49.84 / 96.10	45.33 / 91.03	43.95 / 91.47	49.92 / 95.20	48.54 / 95.18	51.57 / 96.09	49.95 / 96.11

Table 14: Reference-based Chinese-centric evaluation results (spBLEU / XCOMET) of baseline models (Part I) on the FLORES+ benchmark.
Direction	TranslateGemma-4B	TranslateGemma-12B	TranslateGemma-27B	GemmaX2-28-2B	GemmaX2-28-9B	Hunyuan-MT-7B	HY-MT1.5-1.8B	HY-MT1.5-7B	HY-MT2-1.8B	HY-MT2-7B	HY-MT2-30B-A3B	Seed-X-Instruct-7B	Seed-X-PPO-7B	Tower-Plus-2B	Tower-Plus-9B	Tower-Plus-72B
zhs → ar	20.57 / 85.48	22.40 / 90.53	23.94 / 92.21	23.78 / 85.41	28.27 / 90.53	23.10 / 91.34	20.05 / 86.54	22.37 / 91.54	22.22 / 87.65	24.07 / 92.15	26.50 / 92.09	26.36 / 88.77	26.77 / 90.74	- / -	- / -	- / -
ar → zhs	23.42 / 81.50	26.12 / 87.23	28.87 / 88.59	29.88 / 83.94	32.97 / 87.55	22.93 / 88.19	22.40 / 84.78	24.11 / 88.83	23.76 / 84.67	24.10 / 89.41	30.56 / 90.10	31.85 / 86.49	27.59 / 87.27	- / -	- / -	- / -
zhs → az	4.75 / 34.69	15.04 / 71.10	17.28 / 79.83	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
az → zhs	18.51 / 68.43	21.47 / 78.90	23.13 / 81.09	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
zhs → bn	16.84 / 83.05	18.08 / 88.10	20.66 / 89.69	20.33 / 84.02	23.41 / 88.28	18.16 / 87.60	16.49 / 83.25	17.86 / 88.29	17.70 / 84.48	18.86 / 90.62	20.53 / 91.40	- / -	- / -	- / -	- / -	- / -
bn → zhs	21.35 / 74.92	24.93 / 81.79	26.90 / 83.97	26.94 / 78.04	30.01 / 82.77	18.29 / 77.93	18.56 / 76.90	20.32 / 80.24	19.01 / 76.65	20.88 / 83.59	27.33 / 85.93	- / -	- / -	- / -	- / -	- / -
zhs → bg	23.72 / 87.94	25.90 / 93.10	28.28 / 94.30	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
bg → zhs	26.16 / 84.38	28.24 / 87.89	30.11 / 88.89	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
zhs → ca	21.71 / 84.58	25.94 / 90.70	24.19 / 92.08	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
ca → zhs	26.10 / 81.89	28.91 / 87.52	30.81 / 88.52	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
zhs → cs	19.44 / 79.11	22.18 / 87.18	23.52 / 88.99	25.40 / 84.69	28.55 / 89.72	23.53 / 88.14	20.62 / 80.47	22.57 / 88.11	24.81 / 83.93	28.24 / 89.99	28.96 / 91.21	26.13 / 85.35	28.68 / 89.67	23.50 / 82.00	27.47 / 88.88	28.39 / 89.56
cs → zhs	25.91 / 80.51	27.66 / 84.90	29.77 / 86.22	31.14 / 82.51	34.05 / 85.17	24.99 / 87.15	24.04 / 84.15	25.20 / 87.15	26.91 / 83.76	26.93 / 88.42	33.24 / 87.18	32.84 / 82.62	30.17 / 84.50	29.81 / 78.89	33.81 / 83.99	36.21 / 84.79
zhs → zht	11.59 / 94.06	19.39 / 95.16	29.06 / 96.05	- / -	- / -	25.22 / 95.35	23.86 / 94.62	25.86 / 95.60	28.17 / 95.29	26.96 / 96.15	29.47 / 94.65	- / -	- / -	30.81 / 94.26	26.00 / 94.27	29.78 / 94.57
zht → zhs	10.84 / 94.19	31.17 / 95.18	32.75 / 95.39	- / -	- / -	28.87 / 95.83	28.03 / 94.65	28.33 / 95.71	30.76 / 93.62	31.03 / 94.45	29.55 / 92.43	- / -	- / -	33.23 / 93.65	34.43 / 94.28	34.35 / 94.01
zhs → da	22.74 / 91.40	25.52 / 94.77	26.23 / 95.83	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	29.00 / 93.91	30.91 / 95.58	25.54 / 91.99	28.98 / 94.68	29.85 / 95.07
da → zhs	26.93 / 87.00	29.11 / 90.98	30.93 / 91.69	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	33.99 / 89.84	31.16 / 91.23	30.70 / 86.48	34.94 / 89.98	37.03 / 90.85
zhs → de	22.32 / 95.26	24.66 / 97.34	26.45 / 97.78	28.04 / 96.11	30.67 / 97.39	25.36 / 97.37	20.03 / 94.66	24.76 / 97.36	23.12 / 95.98	25.67 / 97.48	28.37 / 98.05	29.03 / 97.08	30.82 / 97.61	24.82 / 95.31	29.04 / 97.23	30.08 / 97.46
de → zhs	26.79 / 87.69	28.18 / 91.89	29.80 / 92.57	32.39 / 89.90	34.64 / 91.81	26.32 / 93.15	25.73 / 91.29	25.85 / 93.26	28.85 / 90.84	28.55 / 93.43	31.23 / 93.94	33.54 / 91.04	29.53 / 92.03	30.39 / 87.43	34.49 / 91.10	36.15 / 92.05
zhs → el	20.10 / 81.89	24.24 / 89.95	26.17 / 91.70	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
el → zhs	22.79 / 79.35	25.82 / 84.52	27.94 / 85.51	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -
zhs → en	27.24 / 95.63	28.90 / 96.71	30.94 / 97.27	34.68 / 96.48	36.57 / 96.96	29.57 / 96.86	23.89 / 96.00	27.48 / 96.82	26.95 / 96.17	28.98 / 97.04	29.82 / 97.68	36.47 / 96.46	33.19 / 96.86	32.02 / 95.53	34.78 / 96.77	36.72 / 97.26
en → zhs	31.02 / 91.99	31.96 / 94.45	33.60 / 95.15	37.94 / 92.88	39.85 / 93.72	31.03 / 96.04	30.57 / 93.81	29.94 / 95.58	35.79 / 93.47	36.01 / 95.64	37.68 / 95.71	37.60 / 94.01	34.09 / 95.28	36.39 / 90.84	39.92 / 93.32	42.21 / 94.51
zhs → tl	18.44 / 82.16	20.95 / 86.47	23.21 / 87.89	23.44 / 83.91	25.74 / 86.88	17.81 / 79.00	16.72 / 80.67	17.21 / 79.27	20.90 / 83.77	24.23 / 88.14	25.07 / 88.47	- / -	- / -	- / -	- / -	- / -
tl → zhs	25.73 / 74.28	27.41 / 80.25	29.86 / 81.63	31.42 / 77.91	34.20 / 80.73	23.22 / 80.73	23.17 / 77.04	23.82 / 80.99	26.39 / 77.64	25.74 / 83.34	32.49 / 83.16	- / -	- / -	- / -	- / -	- / -
zhs → fi	17.40 / 77.32	21.62 / 87.49	23.32 / 90.29	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	27.87 / 85.96	29.19 / 90.04	20.57 / 81.75	25.33 / 89.53	24.80 / 88.02
fi → zhs	24.45 / 80.16	26.60 / 86.33	28.57 / 87.24	- / -	- / -	- / -	- / -	- / -	- / -	- / -	- / -	31.45 / 82.90	28.18 / 84.84	27.73 / 79.77	32.30 / 80.85	33.81 / 85.54
zhs → fr	27.86 / 84.17	30.30 / 88.21	29.75 / 89.78	33.99 / 87.57	36.76 / 90.82	29.40 / 89.32	24.35 / 85.62	28.54 / 89.34	29.19 / 86.69	32.01 / 90.36	33.69 / 91.65	36.43 / 89.25	35.23 / 91.02	30.17 / 84.99	33.70 / 89.90	35.61 / 91.42
fr → zhs	26.80 / 85.25	28.26 / 88.69	29.81 / 89.42	32.53 / 86.39	34.74 / 87.88	26.36 / 90.78	25.47 / 87.64	25.98 / 90.87	29.10 / 87.28	28.36 / 91.08	32.62 / 90.50	33.71 / 86.59	30.29 / 88.72	30.27 / 83.46	34.26 / 87.35	36.22 / 88.39
zhs → he	17.50 / 78.51	22.43 / 89.71	25.07 / 91.72	25.00 / 84.69	28.36 / 90.70	18.51 / 81.85	8.12 / 74.47	19.08 / 83.97	19.57 / 82.23	22.87 / 91.40	25.92 / 92.13	- / -	- / -	- / -	- / -	- / -
he → zhs	24.27 / 82.54	27.10 / 88.67	30.14 / 90.15	30.68 / 86.11	33.60 / 89.02	22.40 / 87.55	21.81 / 83.82	23.40 / 88.08	23.75 / 83.98	24.23 / 90.43	29.09 / 91.74	- / -	- / -	- / -	- / -	- / -

Table 15: Reference-based Chinese-centric evaluation results (spBLEU / XCOMET) of baseline models (Part II) and MiLMMT models on the FLORES+ benchmark.
Direction	Google Translate	NLLB-54.5B	Gemini 2.5 Pro	Gemini 3 Pro	GPT-5	MiLMMT-46-1B-v0.1	MiLMMT-46-1B-v1.0	MiLMMT-46-4B-v0.1	MiLMMT-46-4B-v1.0	MiLMMT-46-12B-v0.1	MiLMMT-46-12B-v1.0
zhs → ar	31.72 / 91.10	27.53 / 86.46	29.47 / 92.89	30.70 / 93.03	26.38 / 92.23	21.96 / 81.00	21.59 / 82.73	26.33 / 88.56	25.33 / 89.73	28.77 / 91.25	27.13 / 92.36
ar → zhs	37.23 / 87.26	21.25 / 70.04	34.46 / 89.09	35.09 / 89.37	32.71 / 89.05	26.16 / 80.95	26.13 / 82.10	30.85 / 85.73	30.42 / 87.01	32.55 / 88.06	32.00 / 89.16
zhs → az	21.35 / 79.59	19.24 / 70.95	22.51 / 86.13	22.90 / 86.77	20.43 / 84.90	16.86 / 66.61	16.48 / 69.88	19.86 / 79.83	18.91 / 81.08	21.21 / 84.04	20.46 / 84.95
az → zhs	30.41 / 79.10	19.32 / 64.19	27.77 / 82.79	28.12 / 82.86	25.42 / 83.07	21.06 / 70.93	20.83 / 73.74	25.17 / 79.29	24.55 / 80.79	26.56 / 81.08	25.76 / 82.83
zhs → bn	28.29 / 86.23	23.67 / 81.38	24.99 / 89.62	25.28 / 90.72	21.08 / 90.00	17.26 / 78.55	17.51 / 81.08	21.89 / 87.10	20.13 / 88.57	23.36 / 89.17	21.52 / 90.90
bn → zhs	34.34 / 82.48	20.01 / 63.34	31.52 / 85.02	32.57 / 85.09	30.31 / 85.45	23.20 / 72.94	23.47 / 75.43	28.46 / 80.47	27.97 / 82.44	30.24 / 83.32	29.62 / 85.00
zhs → bg	36.71 / 94.01	30.60 / 87.40	34.66 / 95.22	35.53 / 95.40	30.27 / 95.01	26.84 / 85.76	26.25 / 87.18	31.48 / 92.71	30.43 / 93.36	33.28 / 94.13	31.90 / 94.58
bg → zhs	37.56 / 86.59	20.07 / 62.52	34.37 / 88.34	35.17 / 88.18	32.55 / 87.96	28.37 / 81.95	28.18 / 83.75	32.38 / 86.17	31.79 / 87.91	33.90 / 87.30	33.13 / 89.11
zhs → ca	34.51 / 91.79	29.58 / 85.53	32.56 / 93.53	34.25 / 93.74	27.94 / 93.75	25.93 / 85.76	25.54 / 87.25	29.86 / 91.47	27.04 / 92.20	32.45 / 92.83	29.26 / 93.63
ca → zhs	38.27 / 85.84	23.32 / 67.61	34.95 / 88.07	35.72 / 87.80	33.38 / 88.15	29.71 / 81.70	29.46 / 83.30	33.30 / 86.21	33.11 / 87.89	34.29 / 87.57	33.77 / 89.37
zhs → cs	32.24 / 90.16	25.63 / 78.74	30.03 / 91.94	31.73 / 92.55	27.70 / 91.77	22.11 / 76.85	21.76 / 80.01	26.64 / 88.21	26.04 / 89.28	29.02 / 90.99	27.70 / 91.09
cs → zhs	37.49 / 83.38	19.60 / 57.22	34.24 / 85.74	34.53 / 85.46	32.31 / 86.04	28.46 / 78.64	28.46 / 80.74	32.24 / 83.61	32.10 / 85.52	33.89 / 85.03	33.21 / 87.11
zhs → zht	30.38 / 94.37	13.14 / 67.33	29.47 / 94.46	29.32 / 94.48	29.23 / 94.43	29.66 / 93.97	28.95 / 94.22	30.71 / 94.92	29.72 / 95.00	30.92 / 95.17	29.65 / 95.72
zht → zhs	28.49 / 91.58	24.10 / 82.35	29.61 / 91.96	29.96 / 92.11	29.56 / 92.15	31.77 / 92.93	32.15 / 93.39	33.74 / 93.91	33.84 / 94.66	34.44 / 94.20	29.34 / 94.67
zhs → da	34.41 / 95.48	28.32 / 90.49	32.07 / 96.75	33.17 / 97.09	29.61 / 97.00	24.93 / 89.35	24.37 / 90.58	29.37 / 94.76	28.43 / 95.27	31.85 / 96.17	30.41 / 96.39
da → zhs	38.86 / 90.50	19.51 / 62.21	35.39 / 91.64	35.93 / 91.39	33.70 / 91.39	30.25 / 86.40	29.91 / 87.30	34.18 / 89.96	33.94 / 91.35	34.76 / 90.93	34.00 / 92.20
zhs → de	33.71 / 97.85	27.15 / 93.92	31.92 / 98.08	32.77 / 98.04	29.21 / 97.89	23.95 / 93.60	23.30 / 94.71	28.45 / 96.90	27.55 / 97.29	30.64 / 97.68	29.22 / 97.92
de → zhs	37.98 / 91.31	21.48 / 71.34	34.80 / 92.50	34.91 / 92.33	32.66 / 92.20	29.24 / 87.59	29.04 / 89.08	32.77 / 90.75	31.89 / 92.09	34.04 / 92.23	33.15 / 93.00
zhs → el	31.35 / 91.24	26.00 / 84.84	31.29 / 92.90	31.95 / 93.14	28.72 / 92.75	22.27 / 79.61	22.55 / 81.50	27.07 / 89.97	26.35 / 90.52	29.29 / 92.08	28.39 / 92.55
el → zhs	35.41 / 82.39	20.76 / 64.25	32.70 / 84.88	33.23 / 84.44	30.77 / 84.92	26.16 / 76.52	25.63 / 78.50	29.71 / 82.27	29.18 / 84.70	32.05 / 84.16	31.11 / 86.04
zhs → en	39.59 / 97.18	36.13 / 95.04	38.26 / 97.28	38.12 / 97.38	34.82 / 97.52	32.44 / 95.14	30.86 / 95.53	35.40 / 96.59	33.76 / 96.79	36.60 / 97.21	34.99 / 97.36
en → zhs	44.05 / 93.84	26.61 / 76.72	39.42 / 94.26	40.26 / 94.43	37.73 / 94.19	35.03 / 90.32	34.23 / 91.75	38.03 / 93.21	36.85 / 94.54	38.98 / 94.11	37.30 / 95.32
zhs → tl	25.62 / 82.81	22.81 / 79.27	26.11 / 88.76	26.56 / 89.09	24.28 / 87.82	20.15 / 77.37	19.36 / 80.15	23.92 / 85.37	22.67 / 87.16	25.21 / 87.31	23.94 / 89.04
tl → zhs	38.06 / 79.55	21.61 / 61.36	34.81 / 81.94	35.80 / 81.73	32.34 / 81.99	27.68 / 73.83	27.17 / 75.28	32.15 / 79.13	31.34 / 81.46	33.62 / 80.86	32.45 / 82.63
zhs → fi	30.34 / 90.74	23.09 / 80.21	30.38 / 93.70	31.99 / 94.01	27.79 / 93.17	18.22 / 72.38	18.02 / 75.88	24.37 / 88.05	23.56 / 88.99	27.70 / 91.28	26.69 / 92.25
fi → zhs	35.43 / 84.37	18.72 / 57.59	33.03 / 86.40	33.75 / 86.81	31.33 / 86.97	26.03 / 78.18	26.02 / 79.88	31.06 / 84.61	30.38 / 85.84	32.53 / 86.08	31.46 / 87.96
zhs → fr	39.17 / 91.05	34.16 / 83.32	37.57 / 91.97	38.91 / 92.35	33.42 / 91.48	30.47 / 83.15	28.47 / 84.27	34.99 / 89.39	31.82 / 90.01	36.63 / 90.98	33.62 / 91.75
fr → zhs	38.27 / 86.91	21.19 / 63.19	34.52 / 89.06	34.94 / 88.71	32.83 / 88.93	29.68 / 83.12	29.66 / 85.07	32.67 / 87.05	31.94 / 88.43	34.00 / 88.04	33.47 / 90.30
zhs → he	34.40 / 90.22	27.04 / 84.83	30.38 / 92.79	32.47 / 92.88	27.35 / 92.58	18.70 / 75.10	19.36 / 77.41	26.33 / 88.70	25.01 / 89.71	29.13 / 91.37	27.85 / 92.54
he → zhs	38.02 / 88.74	19.77 / 65.10	35.10 / 90.40	35.51 / 90.49	33.13 / 89.97	26.47 / 80.89	26.31 / 82.34	31.83 / 87.45	31.59 / 88.62	33.39 / 89.13	32.72 / 90.40

Table 16: Full on-policy distillation results as 46-language. All students are distilled from the fixed MiLMMT-46-12B-v1.0 teacher. WMT24++ cells report reference-free XCOMET / COMETKiwi; each FLORES+ cell reports spBLEU / reference-based XCOMET / reference-free XCOMET / COMETKiwi.
Model	WMT24++	FLORES+
	en → xx	en → xx	xx → en	zh → xx	xx → zh
1B students
RL+OPD λ=1	78.25/75.35	33.87/87.79/90.71/86.56	39.65/91.17/90.65/88.27	21.64/82.24/82.76/77.77	26.77/81.48/67.18/76.93
RL+OPD λ=0.5	78.58/75.69	33.76/87.90/90.90/86.72	39.41/91.24/90.75/88.30	21.63/82.43/82.97/78.00	26.70/81.57/67.50/77.10
RL+OPD λ=0.1	79.37/76.41	33.32/88.38/91.39/87.19	38.56/91.44/91.02/88.36	21.34/82.93/83.66/78.61	26.40/81.89/68.14/77.43
RL+OPD λ=0.05	79.83/76.68	32.66/88.49/91.56/87.29	38.02/91.47/91.11/88.37	21.04/83.00/83.72/78.80	26.04/82.09/68.39/77.84
RL+OPD λ=0.01	80.29/77.01	31.11/88.53/91.74/87.29	37.33/91.45/91.14/88.33	20.47/83.12/83.95/78.97	24.89/82.10/68.74/77.78
RL+OPD λ=0.001	80.42/77.08	29.65/88.37/91.66/87.29	36.30/91.46/91.15/88.24	19.70/82.82/83.82/79.14	24.12/82.05/68.51/77.79
RL+OPD λ=10−4	80.26/76.92	30.03/88.32/91.62/87.19	36.65/91.45/91.11/88.27	19.89/82.82/83.93/79.08	24.18/82.04/69.08/77.97
OPD (PG)	77.67/74.66	33.95/87.19/90.21/86.03	39.78/90.97/90.42/88.19	21.44/81.63/82.13/77.17	26.70/81.07/66.70/76.58
OPD (GKD)	77.42/74.37	32.90/87.12/90.11/85.91	39.54/90.96/90.44/88.15	20.83/81.32/81.77/76.86	26.35/81.02/66.56/76.40
OPD (PG, init. v1.0)	77.86/74.93	33.87/87.45/90.43/86.29	39.58/91.06/90.54/88.23	21.43/81.93/82.40/77.42	26.69/81.23/66.87/76.68
4B students
RL+OPD λ=1	85.85/82.23	37.09/93.34/95.15/91.12	42.70/93.98/92.78/89.41	25.44/89.92/88.56/82.36	30.68/86.67/70.43/79.99
RL+OPD λ=0.5	86.03/82.45	36.81/93.44/95.27/91.27	42.43/94.03/92.86/89.42	25.22/89.96/88.60/82.51	30.43/86.89/70.59/80.17
RL+OPD λ=0.1	86.67/82.95	35.40/93.72/95.57/91.57	41.59/94.11/93.00/89.42	24.55/90.13/88.93/82.85	29.67/87.27/71.50/80.57
RL+OPD λ=0.05	87.03/83.17	34.25/93.74/95.62/91.66	41.22/94.10/93.03/89.41	24.01/90.09/88.97/82.95	29.18/87.30/71.36/80.55
RL+OPD λ=0.01	87.62/83.46	31.76/93.89/95.88/91.77	41.06/94.16/92.99/89.39	23.38/90.19/88.96/83.28	28.19/87.61/71.41/80.66
RL+OPD λ=0.001	87.68/83.57	30.35/93.87/95.88/91.80	40.64/94.13/92.95/89.38	22.70/90.16/89.03/83.45	27.12/87.71/71.98/80.94
RL+OPD λ=10−4	87.74/83.45	30.41/93.85/95.83/91.74	40.27/94.14/92.99/89.36	22.63/90.06/89.01/83.52	26.63/87.63/71.56/80.67
OPD (PG)	85.42/81.86	37.56/93.15/94.97/90.94	42.88/93.91/92.68/89.37	25.46/89.74/88.36/82.13	30.80/86.47/70.01/79.79
OPD (GKD)	85.62/81.96	36.97/93.21/95.01/91.01	42.68/93.94/92.76/89.38	25.28/89.76/88.36/82.25	30.71/86.58/70.08/79.88
OPD (PG, init. v1.0)	85.37/81.85	37.53/93.14/94.97/90.97	42.78/93.89/92.71/89.40	25.47/89.69/88.32/82.09	30.85/86.45/70.08/79.76

실제로 확인된 결과

WMT24++에서 세 모델 크기 평균으로 참조 없는 XCOMET은 2.75점, COMETKiwi는 2.44점 향상되었다(지도학습 모델 대비).
FLORES+에서는 참조 기반 XCOMET 1.17점, 참조 없는 XCOMET 1.41점, COMETKiwi 1.17점이 올랐지만 spBLEU는 1.21점 낮아졌다(지도학습 모델 대비).
MiLMMT-46-12B-v1.0은 WMT24++와 FLORES+ 네 개 방향군 모두에서 Google Translate, Gemini 2.5/3 Pro, GPT-5를 포함한 평가 대상 상용 시스템보다 높은 참조 없는 점수를 기록했다.
MiLMMT-46-1B-v1.0은 파라미터 수가 4분의 1에 불과한데도 TranslateGemma-4B의 모든 WMT24++·FLORES+ 지표를 앞질렀다.
1B, 4B 학생 모델에 온폴리시 증류(RL+OPD, λ=1)를 적용한 결과, MiLMMT-46-v1.0 기준선에 근접했지만(FLORES+ 예: 4B 모델 spBLEU 34.17 대 33.96, 참조 기반 XCOMET 90.82 대 90.91) 이를 일관되게 넘어서지는 못했다.

어디에 쓸 수 있나

정답 번역문 확보가 어려운 저자원 언어 번역 모델을 원문 텍스트만으로 사후 학습(post-training)해 품질을 높이는 데 참고할 수 있다.
강화학습으로 번역 모델이 유창하지만 다른 언어를 내뱉는 오류를 줄이는 언어 판별 보상 설계에 응용할 수 있다.
큰 번역 모델의 품질을 별도 강화학습 없이 작은 모델로 옮기고 싶을 때 온폴리시 증류를 대안으로 검토할 수 있다.
지도학습 모델과 강화학습 모델을 섞는 체크포인트 보간을 통해 어휘 일치도(spBLEU)와 품질 평가 점수 사이의 트레이드오프를 조절하는 실무에 적용할 수 있다.

한계와 남은 검증

평가는 46개 언어와 WMT24++, FLORES+ 두 벤치마크, 그리고 지정된 비교 모델군에 한정되어 있어 다른 언어·도메인에서의 성능은 확인되지 않았다.
강화학습 이후 spBLEU(단어 일치도)가 낮아지는 경향이 있으며, 체크포인트 보간으로 일부만 회복될 뿐 완전히 상쇄되지는 않는다.
온폴리시 증류는 12B 교사 모델이 만든 품질 수준에 근접하지만 이를 넘어서지 못하며, 왜 넘어서지 못하는지에 대한 추가 분석은 향후 과제로 남아 있다.
저자들은 더 견고하고 인간 판단에 가까운 보상 모델 탐색, 멀티모달 다국어 번역으로의 확장, 더 큰 모델 규모에서의 검증을 향후 연구로 계획하고 있다.
평가에 사용된 품질 추정 모델(XCOMET, COMETKiwi) 자체의 편향이나 한계가 결과에 반영될 수 있으며, 이에 대한 별도 검증은 제시되지 않았다.

왜 중요한가

정답 번역문이 부족한 저자원 언어에서도 원문 텍스트만으로 번역 품질을 개선할 수 있다는 것을 보여주며, 이는 다국어 번역 시스템을 만들 때 데이터 확보 부담을 줄이는 방법이 될 수 있다. 또한 모델과 코드를 공개해 다른 연구자나 개발자가 같은 방식을 자신의 언어·도메인에 적용해볼 수 있게 했다.

이 논문의 용어

GRPO(Group Relative Policy Optimization) · 여러 개의 번역 후보를 만들어 서로 비교한 점수 차이를 이용해 모델을 학습시키는 강화학습 방법
참조 없는 평가(reference-free QE) · 정답 번역문 없이 원문과 번역문만 보고 품질을 매기는 자동 평가 모델(XCOMET, COMETKiwi 등)
체크포인트 보간 · 서로 다른 학습 단계에서 저장한 모델 가중치 두 개를 비율대로 섞어 하나의 새 모델을 만드는 방법
온폴리시 증류(on-policy distillation) · 큰 교사 모델이 작은 학생 모델 스스로 만든 번역 결과를 보고 점수를 매겨 학생을 학습시키는 방법
spBLEU · 정답 번역문과 얼마나 단어·구절이 겹치는지를 재는 전통적 번역 품질 지표

저자 · Chris Han, Pengzhi Gao, Pei Fu, Jian Luan

arXiv에서 원문 보기

METAL MEDIA 최신 기사

그림 출처: Chris Han et al., arXiv:2608.10812, arxiv-nonexclusive

정답 번역문 없이도 강화학습으로 번역 AI 품질을 올린 오픈소스 다국어 번역 모델 이야기

MiLMMT-46 사후 학습 파이프라인

무엇을 했나

실제로 확인된 결과

어디에 쓸 수 있나

한계와 남은 검증

왜 중요한가

이 논문의 용어

최신 논문

METAL MEDIA 최신 기사