Voicebox: Artificial Intelligence that Can Replicate the Voices of Your Friends and Family

2023-06-20

The breakthrough in this technology was announced through a published paper. While it has not been widely used yet, you can listen to a demonstration. As AI chatbots and art generators gain popularity, the most prominent players in this industry are striving to stay ahead with their tools. Meta has just launched Voicebox, a text-guided AI voice generator that is powerful and claims to outperform all existing similar products. Voicebox is powerful enough to effortlessly generate speech, just like ChatGPT generates text and Bing or Dall-E 2 creates images. Although Voicebox is not widely available to the public yet, Meta has provided a demonstration to anyone interested in learning about Voicebox. For example, this system can be used for audio editing by content creators, as its voice generation can make audio clips sound more natural. Furthermore, its capabilities are comprehensive enough to intelligently edit noise in speech clips, such as dog barks, and regenerate the sound. One of Voicebox's abilities is to match the audio style of a sample and generate text-to-speech segments. Essentially, visually impaired users can provide Voicebox with a short audio clip of a friend, as short as two seconds, and it can read out their written information using their friend's voice. New generative AI tools can solve tasks through contextual learning, allowing them to handle previously unseen text and generate context and tone correctly, much like a person learns and solves new problems using existing knowledge. The ethical and legal implications of this groundbreaking tool cannot be ignored. Anyone can use a person's voice recording to generate audio clips without permission. In the published paper, Meta claims that a binary classification model can distinguish between real-world speech and Voicebox-generated speech. However, since the system is not public, Meta's claims have not been verified. Meta trained Voicebox with 60,000 hours of English audiobooks and 50,000 hours of multilingual audiobooks in six languages to achieve optimal performance. This training enables it to perform speech denoising, stylization, editing, and generate diverse speech samples without additional training. In a paper published by Meta AI, it is claimed that Voicebox is 20 times faster and easier to understand than Microsoft's VALL-E. In addition to being faster and making fewer mistakes than competitors, Meta claims that Voicebox can convert written text into spoken language in one or multiple languages without the need for separate training for each language. Compared to the previous state-of-the-art model, YourTTS, Voicebox reduces the average word error rate from 10.9% to 5.2% and increases audio similarity from 0.335 to 0.481. Source: https://www.zdnet.com/article/meta-unveils-voicebox-ai-to-replicate-the-voices-of-your-friends-and-loved-ones/

RECENT AI NEWS

RECENT AI TOOLS