Meta launches powerful voice AI: capable of understanding over 4000 languages worldwide

2023-05-24

Meta has developed an AI language model that is no longer as singular as ChatGPT. Meta's Massively Multilingual Speech (MMS) project can recognize over 4,000 spoken languages and generate speech-to-text conversion in over 1,100 languages. Similar to other public AI projects announced by the company, Meta has now open-sourced MMS to help maintain language diversity and encourage researchers to build upon this foundation. "Today, we are publicly sharing our model and code so that others in the research community can build upon our work. Through this effort, we hope to make a small contribution to preserving language diversity in the world." Speech recognition and text-to-speech models typically require training on thousands of hours of audio, accompanied by transcription labels. (Labels are crucial for machine learning as they allow algorithms to correctly classify and "understand" data.) However, for languages not widely used in industrialized countries, many of which may even face the risk of extinction in the coming decades, "this data simply does not exist," as Meta puts it. Meta has employed an unconventional approach to collect audio data: utilizing recordings of religious texts being translated. The company states, "We turned to religious texts, such as the Bible, which have been translated into multiple languages, and their translations have been widely used for text-based language translation research." "These translations come with publicly available audio recordings of people reading these texts." By integrating unlabeled recordings of the Bible and similar texts, Meta's researchers have expanded the model's available languages to over 4,000. If you are like me, this approach may initially surprise you as it sounds like a design heavily biased towards a Christian worldview in AI models. However, Meta clarifies that this is not the case. "While the audio recordings are of religious content, our analysis suggests that this does not bias the model towards generating more religious language," Meta writes. "We believe this is because we used a Connectionist Temporal Classification (CTC) approach, which is more constrained compared to large language models (LLMs) or sequence-to-sequence models." Furthermore, despite most religious recordings being read by male speakers, it does not introduce male bias - it performs equally well with male and female voices. After training an alignment model to make the data more usable, Meta utilized wav2vec 2.0, its "self-supervised speech representation learning" model, to train on unlabeled data. The combination of unconventional data sources and self-supervised speech models has yielded impressive results. "Our findings suggest that the Massively Multilingual Speech model performs well and covers 10 times more languages compared to existing models," Meta states. Specifically, Meta compared MMS with OpenAI's Whisper, and it exceeded expectations. "We found that the model trained on Massively Multilingual Speech data covers 11 times more languages, despite having a word error rate of half." Meta cautions that its new model is not perfect. "For example, the speech-to-text model may misread certain words or phrases, posing a certain risk," the company writes. "Depending on the output, this may result in offense and/or inaccurate language. We still believe that collaboration across the AI community is crucial for the responsible development of AI technology." Now that Meta has released MMS as open-source for research, it hopes to leverage the technology to gradually narrow down the world's languages to a maximum of 100 or fewer, supported by technology. It envisions a world where assistive technologies, TTS, and even VR/AR enable everyone to speak and learn in their mother tongue. Meta states, "We imagine a world where people are encouraged to keep the vitality of their own languages because they can access information and use technology in their own language."

RECENT AI NEWS

RECENT AI TOOLS