Spotify launches AI audio translation feature, enabling podcasters to easily spread content in multiple languages

What if podcasters could instantly speak in another language with just a flip of a switch? That's the premise behind Spotify's new AI-based voice translation feature, which translates podcasts into other languages using the podcaster's voice.

The company has partnered with some podcasters to translate their English shows into Spanish using its new tool, and plans to roll out French and German translations in the coming weeks. The initial batch of shows will come from some big names, including Dax Shepard, Monica Padman, Lex Fridman, Bill Simmons, and Steven Bartlett. Spotify plans to expand its collaboration team, including "The Rewatchables" show from "The Ringer" company and an upcoming show from Trevor Noah.

The core of the translation feature is OpenAI's Whisper, a speech transcription tool that can transcribe English speeches and translate other languages into English. However, Spotify's tool goes beyond just translating speech to text - it can translate podcasts into different languages and reproduce them in the podcaster's own synthesized voice.

"By matching creators' own voices, the voice translation feature allows global listeners to discover and be inspired by new podcasts in a more authentic way than ever before," said Ziad Sultan, Spotify's Head of Personalization.

OpenAI may also be involved in the voice replication aspect of the new feature. The AI company announced today that it is launching a tool that can create "human-like audio" based on text and a few seconds of sample speech. OpenAI stated that it intends to limit the widespread use of this tool due to concerns about safety and privacy.

This may be why Spotify has stated that it is currently testing the translation technology with only a "small group" of podcasters. The company did not provide detailed information on its plans to expand the tool or when to expect a larger-scale rollout.