pixar I had a problem. She had a great idea for a new movie: Elementarybased on characters from the director of The Good Dinosaur, Peter Sohn, but animating the film’s titular elements was proving to be a problem. After all, it’s one thing to draw a crumbling heap of sentient earth, but how do you capture the ethereal nature of fire on screen, and how would a corporeal body made of water work? Can you see through it? Do the eyes just float around?
While some of those questions could be answered with suspension of disbelief, the animators of pixar They thought the fire problem was a real puzzle, especially considering that one of the main characters in their film, Ember, was supposedly made of that substance. They had tools to create flame effects from years of previous animations, but when they tried to shape a character, the results were pretty scary, a mix between Calcifer of Studio Ghibli and the Ghost Rider by Nicolas Cage, but somehow more shocking.
“Our fire fluid simulations are very naturalistic and designed to mimic reality,” says Sanjay Bakshi, visual effects supervisor. With a character like Ember, Bakshi says “it’s really important to focus on facial performance,” but the studio was having trouble balancing the dynamism of fire with the character’s form and sensibility. Paul Kanyuk, Technical Crowd Supervisor at pixarsays that at the beginning, Ember it looked like a ghost or even a demon. “It can look scary if it’s too realistic, like you have a human figure made out of real fire,” he explains.
Even if you can reduce the scary aspect, Sohn says you still need to create something that’s recognizably flaming.
“Fire is naturally very agitated, but if you slow it down, it can turn into something that looks like plasma,” he explains. “It was interesting to compare him with other anthropomorphized characters, because they are all so fantastic and you can do anything with them. If you’re drawing an emotion, there’s no direct equivalence, but everyone knows what fire looks like.”
Basically, Sohn explains, to create Embereach take of Elementary you would need a special effects process, which is not only time consuming but also very expensive.
Fortunately, Kanyuk had an idea. He has been working on crowd animation in pixar since 2005, starting with ratatouille, and has always struggled to find ways to make the clothes of large groups of people look good. While he was trying to solve the problem, he became involved with Siggraph, a community organization dedicated to advancing computer graphics. Around 2016, he found some research from the group on using machine learning to refine cloth simulations and has been trying to master it ever since.
In around 2019, Kanyuk came across a Siggraph Asia article about using Neural Style Transfer (NST), which is the type of artificial intelligence used to make a photo look like a Van Gogh or Picasso work, to move voxels (basically 3D pixels with volume) in the animation, all with the goal of giving a character a certain look. Kanyuk thought NST could help Pixar solve its fire problem, though he told Sohn (who had also joined as the film’s director) that, as with much machine learning, there was only 50 percent probability that it would work. “I told him, ‘I’ll give you five ideas and maybe two of them will work.’ But he said, ‘Let’s all do it,’” says Kanyuk.
The only downside, of course, was that using that kind of machine learning required a huge amount of computing power. After all, performing a complete process on all 1,600 shots Elementary it would be an absolutely monumental task, especially considering that the process required a large number of graphics processors.
“Originally, we didn’t have the resources, so we told [Sohn] which we could probably only do to Ember in close-ups,” says Bakshi. Later, Kanyuk says that the animators realized that if they were using the technology in Emberthey also had to use it on other fire characters, so they wouldn’t stand out as some kind of amorphous (and flaming) blob.
“Requirements kept increasing,” says Kanyuk, “so in the end we achieved a 20x speedup from when we started to when we implemented the technology, taking advantage of the GPUs that everyone in pixar have on their computers. We discovered a way to virtualize the GPUs and using half of it overnight, which dropped the render time for one frame from about five minutes to one second.”
It worked. Ultimately, Kanyuk and everyone else involved in Elementary they were able to render the shots they needed. According to him, Pixar is still “scratching the surface” of what NST can do, “but I’m very excited that we found a use case in Elementary that elevated the type of images we can create.”
For Sohn, it was an opportunity to make the film look the way he wanted it to, while also creating something unlike anything audiences had seen before. It symbolized, he says, one of the things he loves about pixar: the encounter between art and technology, where the latter is an important part of the process, but only one element.
“It’s this combination of the left and right brain, and the use of technology as a tool to help express emotions,” says Sohn, “and in turn we can connect with the technology, instead of it feeling like something new and cold. ”.
Via: Wired
editor’s note: I didn’t think they would have had so many problems for this movie, it’s a shame that it’s doing lousy in reviews, I’m curious but I’ll wait for it to be released in Disney+
#Pixar #latest #film #Elemental