K-文化的一切——从回归到 K-美妆，发送到您的邮箱订阅邮件

METAL MEDIA

A Speech Corpus for Mizo Automatic Speech Recognition: Whisper and SraVaani 1.0 Fine-Tuning with Morphology-Aware Evaluation

arXiv:2608.193612026-08-21

为印度低资源语言米佐语打造语音识别:用17.62小时新数据微调Whisper和SraVaani

研究团队从印度米佐拉姆邦200名说话人处收集并整理了17.62小时的米佐语语音数据,用这些数据分别微调了本不支持米佐语的Whisper模型和已经支持米佐语的印度多语言模型SraVaani 1.0。Whisper-large-v3取得了最低的常规词错误率18.08%,在专为米佐语设计的形态感知评估下进一步降到7.22%。SraVaani 1.0经过微调后,词错误率从58.27%大幅降至29.45%。

METAL MEDIA 解读图

为印度低资源语言米佐语打造语音识别:用17.62小时新数据微调Whisper和SraVaani

  1. 01200名说话人通过网页界面朗读了从报纸和法院判决书翻译来的约8000个米佐语句子,最终整理出8274条句子级录音,总计17.62小时。
  2. 02研究者分别对Whisper的small、medium、large-v3三种规模模型,以及本身已支持米佐语的印度多语言模型SraVaani 1.0进行微调,并用不重叠说话人的训练、验证、测试集进行评估。
  3. 03由于米佐语中形态边界的空格写法可自由变化(同一词素既可连写也可分写),常规词错误率会高估实际错误,研究者因此设计了允许最多合并四个相邻词进行比较的形态感知词错误率。
  4. 04Whisper-large-v3表现最佳,常规词错误率为18.08%,形态感知词错误率为7.22%;SraVaani 1.0零样本测试时词错误率高达58.27%,经米佐语专门微调后降至29.45%,形态感知词错误率为17.93%。
  5. 05错误分析显示,未经微调的SraVaani 1.0有时会输出完全错误的文字系统(如梅泰文或天城文而非米佐文字),并在人名地名识别、声门塞音识别上出现较多错误,微调后这些错误大幅减少。
这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

  1. 200名说话人通过网页界面朗读了从报纸和法院判决书翻译来的约8000个米佐语句子,最终整理出8274条句子级录音,总计17.62小时。
  2. 研究者分别对Whisper的small、medium、large-v3三种规模模型,以及本身已支持米佐语的印度多语言模型SraVaani 1.0进行微调,并用不重叠说话人的训练、验证、测试集进行评估。
  3. 由于米佐语中形态边界的空格写法可自由变化(同一词素既可连写也可分写),常规词错误率会高估实际错误,研究者因此设计了允许最多合并四个相邻词进行比较的形态感知词错误率。
  4. Whisper-large-v3表现最佳,常规词错误率为18.08%,形态感知词错误率为7.22%;SraVaani 1.0零样本测试时词错误率高达58.27%,经米佐语专门微调后降至29.45%,形态感知词错误率为17.93%。
  5. 错误分析显示,未经微调的SraVaani 1.0有时会输出完全错误的文字系统(如梅泰文或天城文而非米佐文字),并在人名地名识别、声门塞音识别上出现较多错误,微调后这些错误大幅减少。
Figure 1: Histogram of duration of speech files in the corpus.
Figure 1: Histogram of duration of speech files in the corpus.
Table 1: Data split for training, validation and testing.
SpeakersSentencesHours
Training184765616.18
Validation114261.02
Testing051920.42
Figure 2: Schematic diagram showing the flow of the experimental protocol.
Figure 2: Schematic diagram showing the flow of the experimental protocol.
Table 2: Durational characteristics of the speech database
Sentences8274
Total duration17.62 hours
Minimum duration0.63 seconds
Maximum duration41.22 seconds
Mean duration7.67 seconds
Median duration6.94 seconds
Table 3: Characteristics of the multilingual ASR models evaluated in this study.
ModelArchitectureParametersPretraining dataMel bins
Whisper-smallEncoder–Decoder Transformer244 M680k h80
Whisper-mediumEncoder–Decoder Transformer769 M680k h80
Whisper-large-v3Encoder–Decoder Transformer1,550 M∼5M h128
SraVaani 1.0FastConformer Hybrid RNNT/CTC∼430 M∼31k h–
Table 4: Fine-tuning configurations used for the ASR models.
ModelOptimizerLREffective batch sizeEpochsSelection
Whisper-smallAdafactor5×10−61620Best val. WER
Whisper-mediumAdafactor5×10−61620Best val. WER
Whisper-large-v3Adafactor5×10−61620Best val. WER
SraVaani 1.0AdamW1×10−41620 + extendedBest val. WER
Table 5: Best epochs and corresponding validation WER of four fine-tuned models.
ModelBest epochValidation WER
Whisper-small Mizo–FT1528.99
Whisper-medium Mizo–FT1326.51
Whisper-large-v3 Mizo–FT1323.00
SraVaani 1.0 Mizo–FT1833.81
Table 6: Results of evaluation on five models.
ModelCER (%)WER (%)MA-WER (%)
Whisper-small Mizo–FT04.8324.0011.49
Whisper-medium Mizo–FT04.0221.6908.87
Whisper-large-v3 Mizo–FT03.2618.0807.22
SraVaani 1.017.7158.2736.27
SraVaani 1.0 Mizo-FT06.9029.4517.93
Table 7: Error distribution by model. Foreign-script outputs are reported as the number of utterances; all other entries indicate the number of error occurrences.
ModelForeign scriptNamesGlottal stopsNumeral transcriptsCode-mix error< t Ω >
Whisper-small Mizo–FTNIL177290
Whisper-medium Mizo–FTNIL186530
Whisper-large-v3 Mizo–FT4 sentences84320
SraVaani 1.021 sentences449194931
SraVaani 1.0 Mizo–FTNIL2462194

为什么重要

这项工作公开发布了语音语料库、微调后的语音识别模型,以及一套适合米佐语特点的评估指标,为米佐语语音技术的后续开发提供了基础。研究还表明,对于形态边界空格写法灵活的语言,常规词错误率可能高估真实识别质量,这对使用罗马字母的其他藏缅语族语言评估也有参考价值。

本文术语

  • 词错误率(WER) · 通过比较识别结果与参考文本中的替换、删除、插入词数来衡量语音识别准确度的标准指标
  • 形态感知词错误率(MA-WER) · 研究者为容忍米佐语形态边界空格写法差异而设计的新指标,允许最多合并四个相邻词后再比较
  • 字符错误率(CER) · 与词错误率计算方式相同,但以字符而非单词为基本比较单位
  • 零样本评估 · 不对某语言进行额外微调,直接用预训练模型测试其表现的方式
  • Whisper / SraVaani 1.0 · Whisper是基于Transformer的多语言语音识别与翻译模型;SraVaani 1.0是覆盖多种印度语言的多语言语音识别模型

论文原文摘要（英文）

This study reports the development of an Automatic Speech Recognition (ASR) system in Mizo, a low-resource language. The development included collecting 17.62 hours of speech data, curating it, and fine-tuning the Mizo ASR system with three Whisper multilingual models and with the SraVaani 1.0 Indic multilingual model. Whisper-large-v3 achieved the lowest conventional WER (18.08%), while morphology-aware evaluation yielded a WER of 7.22%. Zero-shot evaluation of the SraVaani 1.0 Indic multilingual model yielded a WER of 58.27%, while Mizo-specific fine-tuning reduced the conventional WER to 29.45% and the morphology-aware WER to 17.93%. The results demonstrate that the Whisper model can achieve a substantially low WER, even when adapted to an unseen language. In contrast, SraVaani 1.0 supports the Mizo language in its multilingual model; however, fine-tuning with carefully curated Mizo speech data substantially improves its performance.

作者 · Priyankoo Sarmah, Sanasam Ranbir Singh, Lalhmingmawia

在 arXiv 阅读

最新论文

全部论文 →

METAL MEDIA 最新报道

图片来源： Priyankoo Sarmah et al., arXiv:2608.19361, cc-by-nc-nd-4.0