Alibaba Group Intelligent Computer InstituteResearchers Linrui Tian, ​​Qi Wang, Bang Zhang and Liefeng Bo have developed an artificial intelligence system that allows artificial intelligence to read selected texts and smoothly change facial expressions based on the texts they read. EMO introduced.

Mouth movements change in accordance with the words

from EMO The most striking aspect is not that it makes a photo or image do the talking, we have seen many other applications that do this. The main difference of this artificial intelligence tool is that in addition to the pre-prepared configuration, it can also animate visuals based on sounds. Moreover mouth movements It also changes depending on the text. In other words, the visual is literally converted into video in accordance with the audio.

It is also striking that the artificial intelligence tool adapts to the sound source. ability to adjust the pace. Artificial intelligence, which can understand the difference between speaking quietly and rapping, adjusts the pace of gestures, facial expressions and mouth movements in animations accordingly. Moreover, artificial intelligence can also make animated characters, images created by artificial intelligence, or anime characters talk.

So how does it work?

Researchers say that the artificial intelligence model Essentially it consists of two parts states. One of them identifies the image and creates moving frames based on the reference image. The other identifies the audio file and identifies the key points. After that too Most important points Images are linked to . Artificial intelligence also has two control modules. One ensures that the character in the image remains unchanged, while the other controls the sound. The results from both sides are then combined.