New Chapter for Audiobooks: AI Voice Technology Gives Books Natural Sound Quality

An era of audiobooks has arrived.

Researchers from Microsoft, MIT, and the Gutenberg Project have announced an initiative to apply AI-generated natural speech technology to books such as "After a Few Words" by Randall Garrett and "Zut and Other Parisians".

Automated audiobook production is not a new concept and has been around for many years. However, a new method is detailed in the arXiv preprint "Large-Scale Automated Audiobook Production", which utilizes the latest generation of neural network text-to-speech technology to generate more realistic speech while saving time and costs.

Existing public domain audiobooks are often affected by mechanical readings. This new method will generate more vivid and realistic readings through subtle emotional differences.

Brandon Walsh, a software engineer at Microsoft, stated, "We use an automatic speaker and emotion inference system to dynamically change the reading voice and intonation based on context."

The narrative parts are read by one voice, while the dialogues in the story are read by different voices. The tone and speaking style are determined by a neural inference system.

Walsh said, "This makes paragraphs with multiple characters and emotional dialogues more realistic and engaging."

Users can adjust the voice, pitch, speed, and intonation according to their personal preferences.

The researchers note that they are preparing a live demonstration to allow the public to generate audiobooks with their own voices. Only a small amount of voice samples are needed to generate the complete audio.

Last April, The Wall Street Journal reported that DeepZen Ltd. has been using voice samples of the late actor Edward Herrmann for the narration of many recent audiobooks. Interestingly, Herrmann passed away nearly a decade ago.

However, through generative AI technology, his voice samples are used to accurately construct fluent dialogues that are almost indistinguishable from actual recordings of the deceased actor's voice.

The Gutenberg Project has already published approximately 5,000 books online, totaling 35,000 hours of speech. Anyone can log in and listen to them, and the service is free.

They will soon offer users the option to record their own books. Users can create a voice profile by reading a few sentences. The Gutenberg Project will generate an AI-generated voice that users can listen to immediately.

Users can read the preface or dedication in their own voice and then upload the complete book text. Once completed, users will receive an email with a link to the audiobook.

In the future, when a mother is unable to read bedtime stories to her 7-year-old son due to overtime work, he can simply open his favorite audiobook and hear his mother's comforting voice narrating adventurous stories.

Alternatively, aspiring actors can use their own voices to bring characters to life in Shakespearean plays and quickly create gifts for friends.

And if the legalities align with the participants, who wouldn't choose the voices of Taylor Swift, Arnold Schwarzenegger, or Morgan Freeman to narrate their novels?