Meta has open-sourced a large-scale multilingual speech (MMS) model for artificial intelligence, which supports automatic speech recognition (ASR) and text-to-speech synthesis (TTS) for over 1,100 languages, as well as language identification (LID) for over 4,000 languages. MMS outperforms existing models and covers nearly ten times the number of languages.
Based on the wav2vec model, MMS is pretrained on a dataset containing 491,000 hours of speech in 1,406 languages. This dataset includes existing cross-lingual datasets and a new dataset with 9,345 hours of unlabeled religious text readings, songs, and other speech in 3,860 languages. To fine-tune the ASR and TTS models, Meta used recordings of Bible readings in 1,107 languages, providing labeled cross-lingual speech data. The fine-tuned MMS model can perform ASR and TTS in these 1,107 languages and language identification (LID) in 4,017 languages. According to Meta, the MMS model achieves state-of-the-art performance in ASR and TTS tasks across various languages and significantly outperforms other models in low-resource languages. The open-sourcing of this model is expected to provide researchers and developers with a powerful tool for building speech recognition and synthesis systems in a wide range of languages, including low-resource languages that are often overlooked in traditional commercial applications.
"Many languages in the world are facing the danger of extinction, and the limitations of current speech recognition and generation technologies only accelerate this trend. We envision a world where technology has the opposite effect, encouraging people to access information and use technology in the languages they love, enabling them to better preserve their languages."
Training AI models for speech processing using supervised learning requires large labeled speech datasets, typically consisting of audio recordings paired with corresponding text. For many languages like English, these datasets are easily obtainable; however, collecting large datasets for low-resource languages with very few native speakers may be impossible. Meta's previous XLS-R and NLLB research showed that a single cross-lingual model combined with self-supervised pretraining can perform well in approximately 100 languages with minimal data, even in low-resource languages. Recently, InfoQ reported on OpenAI's Whisper and Google's USM, which also support around 100 languages each.
To enable their model to handle thousands of languages, Meta needed an audio dataset with more languages. The team chose to use audio recordings from Christian sources, providing labeled speech data for over 1,000 languages, with an average of 32 hours per language. Although the recordings for each language are from a single speaker, typically male, researchers found that this introduced minimal bias in the final model. They also found that training the model heavily on religious text did not introduce any biases. Yann LeCun, Chief AI Scientist at Meta, highlighted several highlights of MMS on Twitter, particularly its "word error rate being half of Whisper's." Some users pointed out that the availability of the model is limited by its non-commercial license. Another user highlighted other drawbacks and questioned whether it is truly better than Whisper, stating, "In my tests, it performs worse than Whisper in transcribing text, mishearing words and missing implied punctuation. Additionally, MMS is about 10 times slower than Faster-Whisper. MMS uses 20 GB of RAM, while Whisper uses around 1 GB. Due to these reasons and others, it is quite impractical for people to use in real-world applications. Also, note that you need to specify the language being spoken, whereas Whisper identifies the language for you. Hopefully, these issues will be addressed over time, and OpenAI will have a competitor in this field."
Source: https://www.infoq.com/news/2023/06/meta-mms-speech-ai/
Alphabet Q1 paid subscribers reached 350M, fueled by YouTube & Google One. YouTube ad revenue slightly missed at $9.88B amid Premium growth. Total Q1 revenue hit $109.9B, beating forecasts.
Anthropic releases Claude connectors for Adobe, Blender, and Ableton, enabling AI workflow automation and app control. It also sponsors the Blender Development Foundation.
Apple introduces an App Store model splitting 12-month plans into monthly installments. On iOS 26.4+, it lowers upfront costs without reducing total price, adding billing tracking.
Google launches Aadhaar digital IDs in Wallet for India, enabling secure QR/NFC verification. A phased partner rollout targets key sectors, while passport-based credentials expand globally.
YouTube tests "Ask YouTube," an AI tool blending text and video for complex queries. Initially for US Premium users, it supports interactive follow-ups, expanding Google's AI search.
Google grants US DoD classified AI access, joining OpenAI and xAI in defense deals amid Anthropic's dispute. Contractual safeguards against surveillance and lethal weapons remain unenforceable.
Mistral AI launches Workflows, a Temporal-powered orchestration platform for secure, Python-driven enterprise AI automation featuring human-in-the-loop approvals.