購読する How To
The phrase “Fix it in post!” has long become a curse, a meme, a horror line, and something that most filmmakers avoid saying at all costs. Yet, editors still find themselves tackling problems that need quick fixing, and often. Random logos, unwanted elements in the shot, faces of people who didn’t sign the release… Especially on smaller productions, like a corporate film or a YouTube video, they become the editor’s responsibility. In the past, some of these footage issues were unsolvable without elaborate VFX skills. Nowadays, we have AI. Or, to be precise, AI masking.
Of course, it’s not a replacement for proper shoot planning or for the expertise of a professional VFX compositor. Yet, for everyday situations where a shot needs a little repair or polish, AI video masking is one of the useful tools at your disposal. Let’s look at a few examples of where and how it’s helpful.
For that, we’ll be using lessons from our recent MZed course “The Efficient Filmmaker: AI Video Masking for the Rest of Us.” There, we explore these tools in detail and include thorough, step-by-step guides. So, if you are completely new to the topic, check it out!
AI masking applications: general approach
Selecting something in your shot using AI is not a one-button task. However, it’s a great deal easier than rotoscoping an object frame-by-frame. So, let’s try it! For simplicity, in this article, I’ll be using DaVinci Resolve’s Magic Mask. If you edit in a different NLE, no worries: It most likely has a similar tool with a similar process. (For instance, we discuss Magnetic Mask and Roto Brush in the course as well).
Before applying the AI mask, let’s evaluate the material we are dealing with. Low-contrast footage can be challenging for AI, so it makes sense to either go for it when your color correction is finished or to prepare a temporary grade on a separate node. Boosting contrast, brightness, and saturation can help you get better results. All done? Go to the Color Page, find an icon with a human silhouette on the central panel below, and click on it.
See this new area with the settings that appeared? These can help you to refine the selection. But first, we’ll need to add some dots on the subject to include all the required elements in the matte. Generally, I recommend using the “Better” mode, especially if you work with people or on fine details.
Once the mask is generated, refine it with additional points, or use the negative qualifier to remove unwanted areas. After that, we want to track it, and here it’s important to find a moment when your subject is fully visible (with all the limbs, for example). Go to this moment in your clip, and then click on the arrows to track forward and backward.
I also recommend experimenting with all of the sliders and settings to get a feel for how they affect a created mask. Once the selection is complete, there are several things we can use it for, including precise color grading or cool text title effects, for example. In this article, though, we concentrate on quick fixes, so below you’ll find a bunch of everyday application examples.
Blurring sensitive content with AI masking
Faces, license plates, phone screens … these can all ruin an otherwise perfect shot if you don’t have permission to show them. (If we don’t see them clearly, though, it is okay.) So, to fix this problem, you can drag and drop a suitable effect on a node with our earlier-created AI mask. In DaVinci Resolve, such effects could be a Mosaic Blur or Defocus, to name a few.
Here’s a result with a Mosaic Blur. A tiny tip: Uncheck the “Blur Alpha” field in the effect settings, and your blur will match the outline of the masked area more precisely.
If you want to expand the selection slightly and soften its edges, you can also increase the “Clean White” setting in the mask options. And yes, I’m aware that it’s neither an elegant nor a fancy look. However, if we’re talking about a documentary or other real-life material where something important happens in the story and you don’t have the possibility of obtaining additional permission, it’s definitely a quick fix.
Removing unwanted objects
Sometimes it’s not a person you have to blur, but a distracting sticker, trash can, or colourful sign in the corner that draws all the attention and needs to be removed. The most frequent scenario, which I bet happens to every editor once in a while, is when they have to hide a logo of a random company that has nothing to do with the video.
Resolve has an Object Removal effect that works paired with Magic Mask and is also an AI-enhanced feature. The process is fairly similar to the one we discussed before, but with some nuances.
- Mask and track the object you want to remove.
- Apply the Object Removal effect to your mask node.
- Hit “Scene Analysis” and then “Build Clean Plate.”
- Fine-tune with the Clean White/Black sliders for smoother edges.
It won’t always give you a flawless result. Most of the time, you’ll be able to see the traces of where the distracting element used to be. However, often this workflow is good enough that nobody else will notice the masking unless they pause the clip and scrutinize it.
AI masking for roto
In the demos so far, we have only selected an object in the shot and added effects to it. However, you can also use AI masking to isolate a subject, place it on a different background, or utilize it as a track matte for a creative overlay. For that, you need to right-click on the empty space in the node area and choose “Add Alpha Output” from the drop-down menu. A blue dot will appear within the tab, and now you only have to connect your mask node to it (by dragging a dashed line from the blue box on its side to this new dot).
When I studied at the university, we had a course on Introduction to VFX. I vividly remember how my classmate rotoscoped a person in the shot and placed another video layer on top of it (with some natural landscapes), creating an abstract and even poetic shot. He had to go through the selection process frame by frame, adjusting the mask by hand each time. I thought I would never be able to do something beautiful like this. Yet, now, it’s possible within two clicks. After you’ve masked, tracked, and isolated your subject, invert the mask by clicking on a sphere icon in its settings. Now your subject acts as a living “window” for whatever you want to show inside their silhouette. Add a graphic or video layer beneath the clip on the Editing page, and enjoy the result.
AI masking: Conclusion
Naturally, these are just a couple of ways to implement AI masking in your editing and color grading workflows. In reality, the creative possibilities are endless. We discuss many additional applications in the course “AI Video Masking for the Rest of Us” on MZed.com.
Overall, AI masking is helpful for editors working under tight deadlines or on smaller projects. However, please remember that it’s not a replacement for trained professionals. Complex compositing, high-end visual effects, or carefully crafted lighting should still be handled by specialists, in my opinion. We should think of AI masking as a supportive tool – it speeds up routine work, expands creative possibilities, and frees up your time for the decisions that really matter.
What else do you get as an MZed Pro member?
As an MZed Pro member, you can access hundreds of hours of filmmaking education. Plus, we’re constantly adding more courses (several are in production right now).
Starting at just $29/month (billed at $349 in the first year, $199 from the second year, or billed monthly at $49), here’s what you’ll get:
- 60+ courses, over 800+ high-quality lessons.
- Highly produced courses from educators with decades of experience and awards, including a Pulitzer Prize and an Academy Award.
- Unlimited access to stream all content during the 12 months.
- Offline download and viewing with the MZed iOS app.
- Discounts to ARRI Academy online courses, exclusively on MZed.
- Most of our courses provide an industry-recognized certificate upon completion.
- Purchasing the courses outright would cost over $9,500.
- Course topics include cinematography, directing, lighting, cameras and lenses, producing, indie filmmaking, writing, editing, color grading, audio, time-lapse, pitch decks, and more.
- 7-day money-back guarantee if you decide it’s not for you (on annual billing only).
Full disclosure: MZed is owned by CineD.
Join MZed Pro now and start watching today!
Feature image source: MZed.
I’d love to hear about your experience! How often do you use AI masking in your projects (if you already do)? Which of its possibilities do you rely on most often? Do you see AI masking more as a safety net or as a creative tool? Let’s exchange some thoughts in the comments!
