Impressive Artificial Intelligence program that creates faces from sound

Technology continues to grow by leaps and bounds, leveraging various areas to discover new capabilities and features. One of them is power. “reconstruct” a person’s face through an audio track.

Study Speech2Face Presented at a Vision and Recognition Models conference in 2019 Decipher what a person looks like through short audio segments.

The document states that the goal of researchers Tae-Hyun On, Tali Dekel, Changil Kim, Inbar Mosseri, William T. Freeman, and Michael Rubinstein from the MIT Science and Research Program is not to reconstruct people’s faces in the same way, but to have an image with physical characteristics related to the sound analyzed.

used to achieve this designed and trained a deep neural network It analyzed millions of videos from YouTube where people were talking. Model learned during training associate sounds with facesthat allows you to produce images with physical characteristics similar to speakersby including I age, gender and ethnicity.

Education is supervised and harmony of faces and voices From Internet videos without the need to model the detailed physical features of the face.

“Our reconstructions from direct voice reveal the relationships between faces and voices. We numerically evaluate and quantify how and in what way our soundten Speech2Face reconstructions resemble actual images of speakers’ faces.”

They explained that because this work could have sensitive aspects due to ethnicity as well as privacy, no particular physical aspect was added to the re-creation of faces and ensured, like anything else. system machine learning, this evolves over time because it increases the knowledge library with each use.

Although the evidence shown suggests that Speech2Face has lots of matches between faces and voiceshe also had some defects where ethnicity, age or gender did not match the sound sample used.

The model is designed to present Statistical correlations between facial features and voice. It should be noted that AI learned from YouTube videos, which do not represent a real sample of the world’s population, shows inconsistencies with education data in some languages, for example.

In this sense, the study itself recommends that, at the end of its conclusions, those who decide to explore and modernize the system consider a wider sample of people and voices so that they machine learning have a wider repertoire of face matching and recreation.

The program was also able to recreate the sound from the cartoons, which is incredibly similar to the sounds in the analyzed sounds.

Because this technology can also be used for malicious purposes, face reconstruction only keeps the one closest to the person and does not give a full face, as this can be an issue for their privacy. Even so, it’s surprising what the technology can do from sound samples.