Researchers have found a method to use machine learning to change facial expressions in videos.
This method can even edit eye movements, head motions etc.
It does all this with a surprising amount of detail in the output.
Artificial intelligence has gained a substantial amount of traction in the world recently. And it is very well deserving of that attention. From self driving cars, to picking up and dropping objects and to even seeing through walls. And in sensitive cases like healthcare, it can be of great help to humans. But the application of AI you will be reading about in this case might evoke a mixture of humour, awe and terror (dont worry you don’t have to freak out yet!)
Researchers have successfully been able to transfer full 3 dimensional head poses, expressions, eye motions etc. from the face of one actor to another. The results are quite funny actually.
How this works?
The video that needs to be tweaked is given as input to the algorithm. The algorithm first tracks the source and target (actors in this case) and uses facial reconstruction approach. The resulting output represents dimensions such as pose of the head, facial expressions and motion of the eyes. And the level of detail is insane.
The core of the algorithm consists of a deep neural network with a space-time architecture. The level of detail in the results has been achieved via adversarial training. This requires only few minutes of training on the input video. With the ability to freely recombine source and target parameters, the developers have been able to demonstrate a large variety of video rewrite applications without explicitly modelling hair, body or background.
The researchers compared their algorithm to previously made algorithms to show how well it works. Previously nobody had been able to blend the background and reconstruct minute details like eye blinking, head movements, etc.
Additionally the algorithm allows you to edit the input video as well. Change the shape of face, expand their facial expressions, remove hair, close one eye, etc.
Check out the video below to see how exactly this works
So Let’s Summarise:
Some people might consider this unethical and wrong, and they are right to think so as this technology can be used to doctor misleading and false information, in the media especially. But some people might consider this funny and promising. We see this more as a reflection of the myriad of possibilities that machine learning opens up and how it can truly change the world we live in. There are endless possibilities that can come out this technology!