OpenAI’s new voice engine can imitate voices based on a 15-second video clip. The tool is being tested by a limited group and is not yet generally available.

Since 2022, OpenAI has been working on Voice Engine, a new tool that can generate a voice similar to the original speaker based on text input and a 15-second audio sample. The company shared some preliminary insights and results from a preview of the voice engine in a blog, but is not making them available for now.

Clone voices

According to OpenAI, the new Voice Engine tool can fully mimic a voice based on text input, a 15-second video clip, taking intonation and emotions into account. It’s not yet clear what data the model is based on, although openAI told TechCrunch that the Voice Engine model is trained on a mix of licensed and public data.

OpenAI gave some previews of what the new AI model has to offer in a blog post. The tool can serve as a reading aid for non-readers or children and can quickly translate content such as videos and podcasts. In addition, it also offers support for people with voice loss or non-verbal people.

Test phase

This new tool is very vulnerable to misuse. Due to the implications, the company will not make this generally available yet. To solve this problem, OpenAI is working on a watermark to be added to AI-based voices and also wants to create a list of voices that cannot be imitated.

OpenAI is testing the tool with a limited number of testers who have previously signed a declaration that they are not allowed to generate votes without the consent of the data subject.