AI is a tool to help CGI and VFX artists. For better future of VFX, we need to cultivate the collaborative approach, not the competition. AI will always fall short in the decision making process. Only the human artist will have an unique perspective and creative touch which can resonate with the story and overall narrative of the project.
In this blog article, we will discuss in detail regarding why AI cannot replace VFX artists.
The current and future state of VFX is very promising. The global VFX market is projected to reach a market value of USD 59.3 billion by 2027. And, undoubtedly, AI will play a major role here. The recent WGA strike did put a dent, but it will soon take a backdoor. There will a huge work of 3D Animation and visual effects will come in a market, due to the backlog of the global scale.
At the same time, with the advancements of AI and ML in the AVGC industry, there has been a burning debate on the topic of ‘Will VFX artists be replaced by AI?’.
The answer is – Big ‘NO’.
In this case study article, we will discuss it thoroughly.
Why AI cannot replace VFX artists?
While AI helps improve the quality and speed of 3D and visual effects work, it won’t replace human artists. In fact, the demand for skilled CGI and VFX artists is expected to grow. AI is best used as a tool to assist artists, rather than replace them.
The creative and technical aspects of VFX require a human touch, making human artists essential in the process. The visual effects artist is not only a software operator. He/she brings wealth of experience and artistic touch to the final render output, which is not possible by any AI and ML based tools.
1. VFX AI tools cannot give consistency
AI is merely a tool. You can think it as a freelancer artist who is doing you gig in cheaper price. The current AI based video tools like Pika labs, Sora, Runway are good to do so.
They bring your imagination to life via proper prompt. Such tools can transform your text commands into stunning visual spectacles. It surely gives boundless creativity, but they can only work on a micro level of visual effects creation – make a trailer / teaser or Instagram reels. They can’t pull out a full length movie or web series. Due to their limitations, they can’t handle such long consistency. It is all because of random nature of AI tools.
Let’s make you understand via an example.
Imagine that we want to make a realistic CGI plane crash sequence. The requirements are as follows.
- One of the engine gets failes in the air and catches fire in the mid air.
- All the passengers are scared.
- The crew is trying their best to control the situation, but plane crashes into a busy street.
- It collide with buildings, cars and lots of humans.
- There must be respective fire, smoke, debris and lots of aftermath.
Let’s do a reality check regarding can AI and ML based VFX tools achieve the final output with technical aspects or not.
- Compositing of 3D renders with live action planes – No
- Match the lighting conditions of the entire sequence – No
- Accurate animation of all the simulations, as per Physics – No
- Maintain a cohesive visual style throughout a film – No
- Maintain texturing as per all situations – No
- Provide various sequences from all possible camera angles – No
- Does AI tools has access to datasets of such incidents – No
These are the major limitations of AI in VFX pipeline and workflow.
So, you understood that why AI cannot replace VFX artists. AI is helpless if it don’t have data. But, it is not a problem with any human. We can conceptualize and execute any kind of phenomena even if we haven’t experienced it by ourself. We are not dependent on past data.
2. Pixel fuc*king (in a different manner)
In the VFX industry, the term ‘Pixel fuc*king’ refers to when someone focuses too much on tiny irrelevant details, that the audience would not notice rather, than the overall composition.
For AI in VFX, pixel fuc*king is coined differently. Even if prompted correctly, you see following errors.
- Twisted body parts
- Weird / unnatural body movements
- Wrong intersections within body and with props
- Lighting jumps
- Undesirable blending of foreground and background objects
AI and ML tools are designed to follow predefined patterns. It cannot tackle an unexpected concept of the filmmaking process. Algorithms cannot adapt it instantly and hence generate wrong outputs.
On a broader term, we call it ‘pixel fuc*king’. You will never like to see movie / commercial which has above mentioned errors. At the end, you always need a human to cleanup all and give an aesthetic output.
Rendering an output with ‘pixel perfection’ is one of the fundamental tasks of a visual effects artist. AI cannot learn the cinematic language that can be derived from team work. It’s very complex process, where AI and ML algorithms will fail. As an example, Battle of the Bastards sequence of GOT S06E09 (Game of Thrones) is one of the best blend of CGI and live action footage. In terms of TV series, it was surreal. Even after 8 years today, it is still remembered as one of the best realistic and believable VFX sequences.
3. Decision making
Can you write an AI code or algorithm which can capture your emotion?
No.
Not possible at all.
AI cannot understand a difference between a good or bad output. It ‘cannot think’ as a human can think. No any AI and ML based VFX tools can quantify any kind of emotional appeal.
Any AI based tool works on prompts, given by human. It is ultimately a human who takes decision. So, AI is just a helper, not more than that. However advanced it may be in the future, it cannot replicate the artistic and creative sense of human intelligence.
Creating final compositing sequences is a collaborative approach. The artist gets adapted with various departments of filmmaking process and finally he/she delivers the complex artistic output. As an example, the black hole of Christopher Nolan‘s Interstellar movie is the master piece. These are coined as iconic scenes of the movie. It is a proper blend of real Physics theories, complex VFX simulations and creative designing.
No doubt, AI is best to create still images, because it has trillions of data available in photo format. But, such is not the case when it comes to the video format. So, AI gets failed if it has to do a long format video output with maintaining a huge number of details.
4. Lack of accurate data / training data
All the latest video making tools are good to create some seconds of cinematics, but fails beyond it. Mostly they are of camera panning and other movements. They don’t generate the accurate 3D models, so can’t achieve the desired camera angle.
As an example, you got a CG lion jump shot through a proper prompt engineering. But, in that output, if you want to take 360 degree shot and from various closeup / long shots, you will not be able to get it.
On the contrary, if you have created a CG lion in any 3D software (Maya, Houdini, 3ds Max etc.), you have all the controls at your disposal. You can do any kind of camera animation as you have the raw file with you. So, Director don’t need to compromise here. With all your creativity and aesthetic sense, you can create an immersive experience that can leave audience speechless. The 3D Richard Parker (Tiger of ‘Life of Pi’) and Caesar (‘Planet of the Apes’ trilogy series) are an excellent examples of it.
All AI tools are trained on vast datasets. But, our AVGC industry is very niche, and hence it doesn’t get all the required data to manipulate. Due to its lack of diversity, the output doesn’t match the expectations of the Director. It gets resulted in weird body anatomy and wrong animations.
As a VFX artist cannot edit the complex AI algorithms and re-write them, he/she has to accept the outputs. On the contrary, if he/she a 3D model, all kinds of modifications can be done in a proper manner. Lack of such data is one of the strongest reason why AI cannot replace VFX artists.
5. AI is good only for repetitious tasks
AI and ML based VFX tools are best for performing labor intensive works like Roto, paint / prep, cleanup, chroma removal, 2D / 3D tracking etc. By analyzing the live action footage, AI algorithms can automatically identify the blue / green screen (chroma) and replace it with the desired background, saving countless hours of manual work.
Beyond it, it cannot take creative decisions like human VFX artists. At the most, it will cause transformation of job roles in our 3D Animation and VFX industry. The human touch in solving complex visual effects shots and sequences will always be there. AI cannot replace the creativity and aesthetics of visual effects artists.
- We cannot use generative AI and ML based tools to create the final output.
- We can use such tools to create concepts, iterations and ideas.
Doing so can give you new perspectives to the overall storytelling process. Such support system will open doors to a new age of storytelling. To talk in reality, many post production studios are already using such tools to enhance the quality of visual effects.
What AI cannot do that a VFX artist can do?
Following are some of the key factors that stand outs the in the race of Human Intelligence Vs Artificial Intelligence (HI vs AI).
- Intuitive thinking
- Artistic approach
- Human judgement
- Creative expertise
- Interpretation of Director’s concept with new perceptions
- Adaptability with concurrent changes
- Complex problem solving techniques
- Cannot establish human / emotional connection (That’s why you see jerks in animation of eye, lips, hair etc.)
- Being an AI tool, it cannot actively discuss with other human teams
In a nut shell, AI cannot replace VFX artists fully as they are at the core of creativity and overall decision making. With their real life experiences, they bring their unique skills to the table. 3D and VFX artists knows how to spark human feelings which can create emotional moments. AI cannot compensate such artistic vision of human CGI professionals. They can do a huge team work and use each others expertise to enhance the overall output.
Future of AI in VFX
You need to understand that future of AI in VFX is strong and going forward, it will be more and more integrated. There is no denial of it. Together, AI and VFX artists can make new creative masterpiece to engage audience. The ultimate solution of this is to keep on upgrading yourself, artistically and technically both. You must continuously improve your overall skills to remain unreplaceable. This positive approach will help you to build a successful career as a CGI and VFX artist.
We need to continuously learn new AI technologies and update ourselves to remain authoritative. Such adaptation will help to be competitive.
The future holds immense demand for VFX artists. Like Covid-19, the WGA strike will also be forgotten, as we will be back on regular track. With the help of many AI and ML based tools, the CGI and VFX artists will produce mesmerizing effects and mind blowing animations. Such collaboration will help small scale studios to push the boundaries and produce amazing visual effects, that were previously not possible.
Even today, human editor is needed for the final touchup of AI generated images. Only after this human intervein, many images become perfect for distribution or production. Such AI generated images are eliminating entry level work only. AI is not creative at all, it works only on algorithms. So, there is never a chance than it can outrun a human mind.
Conclusion
Now you had learned all the reasons why AI cannot replace VFX artists.
At the end, it will be combination of AI (Artificial Intelligence) and HI (Human Intelligence). AI will do all the possible automotive works and human will take care of creative and final executional calls. It will be always a human VFX artist’s role to give output of the desired immersive cinematic experiences.