Table 8: Hyperparameters of the linear CRF. All reported CRF results use a fixed budget of 200 L-BFGS iterations.
Parameter
Value
L1 coefficient (c1)
1.0
L2 coefficient (c2)
10−3
Maximum L-BFGS iterations
200
Validation-based stopping
Not used
All possible transitions
Enabled
Character context window
±2 characters
Inner folds for OOF reliability
5
Reliability smoothing (κ)
5
Table 9: Preliminary hyperparameter search for the supervised BiLSTM–CRF. Emb. denotes the character-embedding dimension, and Hidden denotes the concatenated output dimension of the two directions. The best segmentation F1 is highlighted.
Layers
Emb.
Hidden
Batch
Dropout
LR
F1
2
100
256
512
0.3
5×10−4
0.8636
2
100
128
512
0.3
5×10−4
0.8595
2
100
64
512
0.3
𝟓×𝟏𝟎−𝟒
0.8905
2
100
32
512
0.3
5×10−4
0.8873
2
100
16
512
0.3
5×10−4
0.8423
1
100
64
512
0.3
5×10−4
0.8638
1
100
128
512
0.3
5×10−4
0.8569
3
100
64
512
0.3
5×10−4
0.8659
2
100
64
1024
0.3
5×10−4
0.8829
2
100
64
256
0.3
5×10−4
0.8822
2
100
64
128
0.3
5×10−4
0.8861
2
100
64
64
0.3
5×10−4
0.8870
2
100
64
32
0.3
5×10−4
0.8900
2
100
64
16
0.3
5×10−4
0.8854
2
100
64
32
0.3
1×10−4
0.8847
2
32
64
512
0.3
5×10−4
0.8382
2
64
64
512
0.3
5×10−4
0.8496
2
128
64
512
0.3
5×10−4
0.8814
2
256
64
512
0.3
5×10−4
0.8755
2
100
64
512
0.1
5×10−4
0.8824
2
100
64
512
0.5
5×10−4
0.8870
Table 10: Selected architectural and training parameters of the BiLSTM–CRF. The maximum number of epochs is only an upper bound; the checkpoint with the lowest development loss is restored after early stopping.
Parameter
Selected value
Character embedding size
100
BiLSTM layers
2
Hidden size
32 per direction
BiLSTM output size
64
Emission size
4
Model dropout
0.3
Dictionary dropout
0.2
Parameter initialization
Xavier uniform
Optimizer
Adam
Learning rate
5×10−4
Batch size
512 sentences
Maximum epochs
10,000
Early-stopping patience
3 epochs
Selection criterion
Development NLL
Gradient clipping
5.0
Table 11: Shared architecture of the Transformer–CRF variants. The two external-feature projections are included only in the corresponding fusion models.
Parameter
Value
Character embedding size
192
Maximum sequence length
128
Transformer layers
3
Attention heads
4
Dimension per head
48
Feed-forward size
768
Activation
GELU
Normalization
Pre-LN
Encoder dropout
0.15
Task-head dropout
0.20
Position embeddings
Learned
Dictionary projection
20→32
Distributional projection
8→16
Output layer
Linear + CRF
Table 12: Training parameters of the static Char2Vec initialization.
Char2Vec parameter
Value
Training objective
Skip-gram
Vector size
192
Context window
5
Negative samples
10
Minimum frequency
1
Training epochs
30
Workers
1
Random seed
42
Minimum sequence length
2 characters
Table 13: Masked-language-model pretraining parameters of TangutEncoder. Pretraining completed the full 5,000 steps, and the checkpoint with the lowest validation loss was retained.
MLM parameter
Value
Masking ratio
0.15
Span-masking probability
0.50
Span length
2–4 characters
Replacement strategy
80/10/10
Optimizer
AdamW
Learning rate
3×10−4
Weight decay
0.01
Batch size
32 sequences
Maximum steps
5,000
Warm-up steps
500
Learning-rate schedule
Warm-up then constant
Evaluation interval
200 steps
Early-stopping patience
5 evaluations
Gradient clipping
1.0
Validation split
5 held-out UUIDs
Table 14: Downstream training parameters shared by Transformer–Random, Transformer–Char2Vec and TangutEncoder.
Fine-tuning parameter
Value
Frozen-encoder epochs
3
Optimizer
AdamW
Encoder learning rate
5×10−5
Task-head learning rate
5×10−4
Weight decay
0.01
Batch size
32 sentences
Maximum fine-tuning epochs
10,000
LR-reduction patience
5 epochs
LR-reduction factor
0.3
Early-stopping patience
10 epochs
Selection criterion
Development NLL
Gradient clipping
1.0
Table 15: Comparison of association measures under five-fold cross-validation.
Association features
P
R
F1
OOV-R
IV-R
Freq + dPMI
0.906±0.003
0.905±0.003
0.905±0.003
0.486±0.009
0.945±0.001
Freq + dPMI + Ent
0.907±0.004
0.906±0.004
0.907±0.004
0.491±0.016
0.947±0.002
Freq + Dice
0.905±0.003
0.905±0.004
0.905±0.003
0.485±0.010
0.945±0.002
Freq + Dice + Ent
0.907±0.004
0.906±0.004
0.906±0.004
0.489±0.016
0.946±0.002
Freq + t-score
0.905±0.004
0.904±0.004
0.904±0.004
0.483±0.014
0.944±0.003
Freq + t-score + Ent
0.907±0.003
0.906±0.004
0.907±0.004
0.490±0.013
0.946±0.002
Table 16: Bidirectional maximum matching with different dictionary sources. Values are five-fold means.
Model
P
R
F1
OOV-R
IV-R
Dict-corpus
.845
.890
.867
.219
.954
Dict-dictionary
.787
.728
.756
.666
.733
Dict-all
.821
.740
.779
.665
.748
Table 17: BiLSTM–CRF variants with progressively richer external features.
Model
External features
P
R
F1
OOV-R
IV-R
BiLSTM
0
0.862±0.008
0.873±0.008
0.868±0.008
0.472±0.014
0.912±0.006
BiLSTM+BIE
11
0.885±0.005
0.894±0.004
0.889±0.004
0.546±0.011
0.927±0.004
BiLSTM+BIE+R
14
0.891±0.003
0.898±0.004
0.894±0.004
0.546±0.009
0.931±0.003
BiLSTM+Dictall
17
0.893±0.003
0.900±0.003
0.897±0.003
0.551±0.011
0.933±0.003
+ Dictall + IL
28
0.896±0.003
0.900±0.002
0.898±0.002
0.557±0.009
0.933±0.002
+ Dictall + IL + DD
30
0.897±0.004
0.904±0.003
0.900±0.003
0.554±0.006
0.938±0.002
+ Dictall + IL + DD + Freq
30+2
0.902±0.004
0.909±0.005
0.906±0.004
0.560±0.008
0.943±0.005
+ Dictall + IL + DD + Freq + Coo
30+4
0.902±0.004
0.909±0.004
0.905±0.003
0.563±0.016
0.941±0.004
+ Dictall + IL + DD + Distall
30+8
0.902±0.003
0.909±0.004
0.905±0.003
0.562±0.014
0.942±0.004
Table 18: Genre-level results for the BiLSTM–CRF feature variants.
Secular
Religious
Model
F1
OOV-R
F1
OOV-R
BiLSTM
0.875±0.007
0.498±0.016
0.810±0.021
0.244±0.028
BiLSTM+BIE
0.897±0.005
0.573±0.015
0.830±0.018
0.305±0.043
BiLSTM+BIE+R
0.903±0.004
0.574±0.014
0.829±0.013
0.286±0.053
BiLSTM+Dictall
0.905±0.003
0.579±0.014
0.835±0.016
0.298±0.037
+ Dictall + IL
0.905±0.002
0.586±0.011
0.842±0.018
0.300±0.043
+ Dictall + IL + DD
0.907±0.001
0.586±0.008
0.850±0.021
0.270±0.041
+ Dictall + IL + DD + Freq
0.911±0.003
0.588±0.012
0.862±0.019
0.316±0.056
+ Dictall + IL + DD + Freq + Coo
0.911±0.002
0.591±0.023
0.859±0.026
0.322±0.076
+ Dictall + IL + DD + Distall
0.911±0.001
0.589±0.020
0.860±0.021
0.326±0.051
Table 19: Downstream results of lexicon-aware continued pretraining. Both encoders are evaluated with the same dictionary and distributional features.
Tangut is an extinct language whose script does not explicitly mark word boundaries. We present the first systematic study of Tangut word segmentation using 2,750 expert-annotated segments(31,893 tokens), traditional lexicons, and unlabeled text. Our framework combines a reliability-calibrated lexicon-lattice representation, explicit distributional statistics, and a lightweight character encoder pretrained with MLM. Segment-level five-fold cross-validation shows that lexical and statistical features raise CRF F1 to approximately 0.91. The full TangutEncoder reaches the highest mean F1 (0.911) and improves recall beyond the labeled training vocabulary. These results demonstrate generalization beyond the limited supervised vocabulary across thematically diverse held-out passages, while document-level transfer remains to be evaluated.
