When the main readers realized the profound errors at the end of 2017, it was thanks to a creepy subculture that used the face-swapping tool to insert celebrities into pornography. More recently, experts have worried that the same AI technology can overload false news. But here's evidence that deepfakes could also be used for conventional video editing: Charli XCX's latest music video uses technology as a special effect.
The "1999" video is a perfect use case for deepfakes. In it, Charli and singer Troye Sivan pay homage to several touchstones of the 90s, such as Steve Jobs, the TLC music video "Waterfalls", Titanic, the Nokia 3310, The Sims, and so. At two points, the creators of the video used the same basic deepfake algorithms to paste the faces of Charli and Sivan into dancers imitating the Spice Girls and the Backstreet Boys.
Ryan Staake, the video director and director of the creative studio Pomp & Clout, says the decision to use deepfakes was partly artistic, partly practical. Speaking to The edgeStaake says that the team had a limited window to record the video, so they used deepfakes to avoid having to dress the two singers like each member of the bands.
"When you start thinking about the complexity of having them in and out of the costumes and makeup for each of those characters, it would take five times more," says Staake. "Then, in a way, it was a pragmatic solution. But then, we also started to play with the extravagance and aesthetics of it. It's one of those things where part of the emotion is trying to see if it works. Like, can we use this weird tool of false pornography in a legitimate music video?
Despite its reputation as a pornography tool, the underlying technology behind deepfakes is part of a broad field of legitimate AI study. In recent years, several algorithms have been designed that can adjust and edit realistic videos. Researchers have created tools that can alter people's expressions and turn them into human puppets. The work has also been used in general applications.
Although tools such as Photoshop and After Effects can achieve similar results, machine learning algorithms are often easier for amateurs to use, hence the popularity of the technology and experts' fears that it will be used for misinformation and the deceptions.
Staake says his team used the same basic algorithms that were shared for the first time on Reddit, adjusting them to improve the resolution and editing the resulting footage slightly to match the complexion. As usual in this type of process, algorithms require a lot of data from the subject's face. Staake says that meant getting Charli XCX "to make a series of strange expressions on stage (opening his mouth, blinking) to create a solid foundation for the algorithms."
The result is largely convincing, and Staake says that any imperfection is added to the effect. "I like it to be bad enough to say" Who the hell is that person? "
This seems to be the first time that these algorithms have been used in a music video, and perhaps the first time they have been used for commercial purposes. According to Staake, it's just the beginning. He says his team has used machine learning algorithms in other music videos (like this one, to generate a postcard effect), and Deepfakes will probably be part of their regular editing toolkit.
"We've been playing a lot with him internally, learning how to use it, where it works best," he says. "I think we're definitely going to see more of this, it's an absolutely incredible tool."