Microsoft's FastSpeech AI Accelerates Natural Voice Generation

2019-12-13

The sound generated by the state-of-the-art text-to-speech conversion model is almost indistinguishable from human voice. These models support neural voices provided by Google Assistant, as well as news anchor voices offered by recent services like Alexa and Amazon Polly. However, most models share the same synthesis method, which involves generating a mel-spectrogram representation of the text and then using a vocoder to synthesize speech. As a result, these models have a common drawback - slow inference of mel-spectrograms, leading to word repetitions or omissions. To address this issue, researchers from Microsoft and Zhejiang University have developed a novel machine learning model called FastSpeech. According to a paper presented by the development team at the NeurIPS 2019 conference in Vancouver, this model features a unique architecture that not only improves performance but also eliminates the problem of word repetitions or omissions. FastSpeech includes a length regulator that adjusts the differences between mel-spectrogram sequences and phoneme sequences. Since the length of phoneme sequences is always shorter than that of mel-spectrogram sequences, multiple mel-spectrograms correspond to a single phoneme. The length regulator then extends the phoneme sequence based on duration, matching it with the length of the mel-spectrogram sequence. (An additional duration prediction component determines the duration of each phoneme.) Finally, the speed of speech is adjusted proportionally by increasing or decreasing the number of mel-spectrograms aligned with phonemes or the duration of phonemes. To validate the effectiveness of FastSpeech, the researchers tested it with the open-source LJ Speech dataset and corresponding texts. After randomly dividing the corpus into 12,500 training samples, 300 validation samples, and 300 test samples, they conducted a series of detailed evaluations and tests on speech quality, robustness, and more. The team reported that FastSpeech achieves a quality comparable to Google's Tacotron 2 text-to-speech conversion model and significantly outperforms leading transformer-based models in terms of robustness, with an effective error rate of 0% and a baseline error rate of 34%. Furthermore, it is capable of increasing the speed of generating speech from 0.5x to 1.5x without sacrificing accuracy.

RECENT AI NEWS

RECENT AI TOOLS