A recent paper published on arXiv cs.CL introduces Myovox, a system designed to decode open-vocabulary English text from surface electromyography (sEMG) signals recorded from facial muscles during vocalized speech. The research demonstrates a reduction in the word error rate (WER) on the emg2speech General Corpus.
Key Points
- Myovox decodes open-vocabulary English text from 31-channel surface electromyography (sEMG) signals.
- The system processes sEMG data recorded from facial muscles during vocalized speech.
- It reduces the word error rate on the emg2speech General Corpus from a published 51.17% to 18.53%.
- The initial baseline, after recovering missing decode settings, achieved a 40.63% WER and 39.02% phone error rate (PER).
- A bidirectional Conformer encoder, trained with cross-modal distillation against WavLM-Large layer-9 features, achieved 26.14% WER and 22.34% PER.
- Ensembling two acoustic models and reranking with a QLoRA-fine-tuned 7B language model further reduced the WER to 18.53%.
- This 18.53% WER is reported as the best result on the emg2speech General Corpus.
Context
The Myovox system improves upon existing methods for sEMG-to-text conversion. According to the arXiv paper, the process involved three distinct steps, each measured in isolation. The first step focused on reproducing the baseline performance by recovering open-vocabulary decode settings, achieving a 40.63% WER. The second step involved replacing a causal encoder with a bidirectional Conformer, which was trained using a four-term cross-modal distillation technique against parallel audio's WavLM-Large layer-9 features. This step reduced the WER to 26.14%. The final improvement came from ensembling two acoustic models, unioning their multi-scale n-best lists, and reranking the results using a QLoRA-fine-tuned 7B language model, leading to the 18.53% WER.
Why It Matters
This research demonstrates a significant advancement in decoding speech directly from muscle signals, which could inform the development of assistive communication technologies. The methodology highlights the impact of specific architectural choices and training techniques on reducing error rates in sEMG-to-text systems.
What To Do
- Note the specific techniques used for error reduction, such as cross-modal distillation and language model reranking.
- Compare the reported word error rates with other sEMG-to-text research, acknowledging the corpus-specific nature of the results.
- Watch for further research that applies these methods to other sEMG corpora or explores the negative results mentioned in the paper.