Masking is one of the most effective tools used in After Effects CS5. Masking can reveal or hide portions of an image you may not want an audience to see. This tool can cut out the portion of an alpha channel you may not want visible throughout certain spots of your composition. You can also animate a mask to change shapes over time and use different mask modes to add, subtract or invert a layer.
There are several ways of creating mask shapes. The most effective way of doing this is to use the shape and pen tools in the tools menu. However you can also create mask shapes in other programs such as Illustrator. For the purpose of this exercise however we’ll be focusing on the masking tools contained within After Effects.
The shape tool is most useful for drawing simple mask shapes such as rectangles, circles, etc which can then be altered to fit a specific shape in the composition. Pen tools are a bit more complicated to use but offer the best way of fine tuning your mask. In case you’re confused, the same tools that are used for creating shapes are also used for creating masks. The major difference is that if you don’t have a layer highlighted in the timeline panel (photo a) and draw a shape in the layer panel then you’ll draw a shape. But if you want to draw a mask, then you must select the layer receiving the mask and then draw the mask in the layer panel (photo b)
For example, in Photo B above we decided to draw a circular shaped mask over a portion of the image. We highlighted the layer (circled in red) then selected the circle shape tool and then began drawing the mask shape we wanted in the layer panel. Notice how automatically it cuts out the rest of the image. This is “masking” at work – a way of cutting out a portion of the image. Of course, you can also select the pen tool and draw the mask you want in your composition. In the following examples we selected the pen tool and drew a unique shape around part of the image. The pen tool can take some practice to figure out but the easiest way to use this tool is to begin drawing the shape you want and then close the path by clicking the first vertex point (the first point of the mask you started to draw). Then afterwards you can use the various pen options and selection tool to adjust the mask shape.
After you’ve done drawing your mask, switch back to the selection tool at the top of the tools panel and look beneath the layer’s options (photo below) in the timeline panel to see the Mask’s properties: Mask Path, Mask Feather, Mask Opacity and Mask Expansion. So far, you’ve drawn the mask path which can be altered by keyframing as you go along. For now though, we’re going to focus on the remaining 3 properties.
Mask Feather: A mask’s outlines can be feathered creating anything from an anti-aliased edge to a softer edge. In photo A below we extended the mask feather property to 99 pixels. Notice how the edges of the mask now appear softer, emphasizing more of the subject in the mask. But if we were to increase the mask feather even more it would show more of the outside of the mask (Photo B)
Mask Expansion: Mask expansion enables you to expand or contract the mask shape you created and is represented in pixels. It’s most useful when you have a soft feather and too much of the mask shape is cut – in this case you can expand the mask with a higher value. In the below image, we used the original feather setting from PHOTO B above but increased the mask expansion so that more of the mask expands outward showing more of the original image.
: Each mask you create has it’s own individual opacity to control the mask only. This doesn’t correspond with the layer’s opacity setting which only controls opacity settings for the entire layer under the layer’s transform settings. For example, the photo below shows 3 different mask shapes drawn on the image. Each one has a different opacity setting. We also renamed each mask by clicking the mask name and pressing enter on the keyboard to help with identifying the different masks.
Another useful way of categorizing masks is to use color coding. Notice how each mask has a different color outline in the composition panel to help distinguish between them. They also each have their color box in the timeline panel. This can be set in the timeline panel by clicking the color swatch beside each mask and selecting a color in the ‘Mask Color’ panel that opens up. This will help you avoid confusion when working on layers that contain multiple masks.
Use Mask Feather with Mask Expansion to achieve your desired look. Use Mask Opacity if you have multiple masks on an image and need to vary the opacity level of each (this is also effective when used with mask modes such as add, subtract, intersect, etc)
The composition above is actually a composite of 3 layers placed over each other and stacked in the timeline panel. We took the original layer and kept the original 3 mask shapes we drew earlier in the Mask Opacity example. By varying the opacity levels of all 3 masks and adding different mask modes (Intersect, Lighten, Add) we were able to create a unique image. We also played around with the Mask Feather and Mask Expansion levels for each mask to blend it with the original layer.
Overall masking is a useful way of creatively boosting your work in After Effects. It can be used to stylistically treat your work but can also be useful to cover up errors in footage (such as if you want to cover up something in a shot you don’t want a user to see). There are many other uses of masking that we will get to in later articles but for now play around with combining multiple images with masks to reveal/hide portions of a composition in fun and innovating ways.