Meta has proposed Voicebox, a generative AI model that converts text to audio.

It took a while, but Meta has now officially introduced Voicebox. This generative AI model converts text to audio.

What

The American company defines its model as a “non-autoregressive flow-matching model that is trained to generate spoken word provided the required audio context and text is provided”.

Training

Voicebox has been trained with over fifty thousand hours of audio. Meta used many spoken texts and public audio book transcripts in English, French, Spanish, German, Polish and Portuguese.

According to the researchers, this variety of data would allow the model to generate more content that leads to a real conversation, regardless of the language each participant speaks. Voicebox first learned to predict parts of spoken text based on the text that came before and after it.

Possibilities

The model can also actively adjust the tone. This allows the system to eliminate background noise and even correct mispronounced words.

However, you do not have to worry too much about applications for the time being. Both the app and the source code of Voicebox are not yet publicly available. The reason for this is possible abuse, according to Meta.

Things haven’t always been shining brightly for Mark Zuckerberg’s company lately. Earlier this month, even more emphasis was placed on the importance of AI to reassure remaining employees after numerous layoffs. Additionally, Meta was fined last month for GDPR violations.