Compositing is taking different elements and merging them together into one. There are many techniques to compositing, including chroma-keying and masks. This time around, we’re going to take a look at the Roto Brush Tool in Adobe After Effects.
First off, what is rotoscoping? I went to Wikipedia for their definition and a little history: “Originally, recorded live-action film images were projected onto a frosted glass panel and re-drawn by an animator. This projection equipment is called a rotoscope, although this device has been replaced by computers in recent years.”
For our modern purposes, rotoscoping means drawing a matte around an object or group of objects in a shot, to separate them from the rest of the frame. The results can then have different foregrounds / backgrounds added to them to create a composite. The Roto Brush Tool in After Effects let’s artists quickly make complex mattes around objects, think a person or other complex shape, in order to cut them out and separate them from their original frame without having to freehand draw it for each frame.
Couldn’t you do this by shooting your subject against a green screen? Well yes, unless you have multiple objects in the shot that have to interact with each other.
Which brings us to our case-in-point – people. I worked on a short film not too long ago called Hellevator, from MPS Digital Studios. In the movie, a character kills another man by stabbing him with an ink pen, proving the pen can be mightier than the sword. The production team had decided to do the entire movie against a green screen and to add in blood effects digitally in post. My job for that particular scene was to add in blood from the wound. The blood spatter needed to fly across the screen, but go in front of the victim and behind the protagonist.
Hellevator was written by comic-book scribe David Michelinie, so the director decided on a comic book look for the film. As such, we could be a little over-the-top in our use of blood, not to mention the overall color and feel of the short.
I used some pre-made blood sprays from Detonation Films. I had run it through RE:Vision Effect’s Twixtor. The original footage was in 24p, while the blood spray was in 30p. I used Twixtor to give the 24p conversion a more organic natural look than I was getting from just a straight 24p conversion from After Effects. I imported the resulting footage into my composition.
I put my shots in order on the timeline: background plate first, then my foreground shot. I went ahead and chroma-keyed my background in using the excellent Keylight plug-in that ships with After Effects. above my foreground was the blood, and finally on top a copy of my chroma-keyed foreground shot. I did this to make a sandwich of layers – background on the bottom, then the foreground shot of the actors, then blood, and finally what would be the matted foreground of my protagonist on top. The thing to remember about After Effects, whatever is lowest in the timeline will be the bottom layer of your shot. Whatever is the top will be the highest, and therefore the most visible. I rename my top layer from 0050LU.mov to 0050LUtop.mov, just to keep from any confusion later. Next up, I used the Roto Brush tool on the top foreground shot.
The Roto Brush tool is located in the toolbar above your Project and Comp windows. It resembles a paintbrush tracing a human figure. I click that button, then go down to my timeline and double-click the layer I want to work on. In my Comp window, I get a new tab that says Layer, and gives the name of the layer – in this case 0050LUtop.mov, the name of my top layer. Roto Brush will only work on a layer, not a full Composition window or a Footage window.
Using it is fairly straight-forward. I go to the first frame I want to make a matte for. My brush is a small green circle. I make a small line inside the object the object I want to matte, in this case my protagonist’s arm. A pink/purple line forms around most of my protagonist’s arm (you can change the outline color if need be). I go through and add another small line, more of the arm is now selected. Whoops, a little too much was selected. I hit the ALT key and my cursor turns to red, for subtract. I add a small line next to the part that was selected that I did not want. I go through frame by frame and make touch-ups to the matte. All this tools is is making small lines where you want it, and making small lines where it overshoots for corrections. Yes, it takes a while, but in the end the result is worth it, much easier than drawing a matte by hand frame-by-frame.
As I do this, there are a few small tools that can help me see what my final outcome will be.
There is a toggle to show the Alpha Channel of my selections.
The next one will toggle the Alpha Boundary, showing me just what will be left after my Roto Brush cuts the picture.
The last one toggles Alpha Overlays, leaving whatever will be removed from the picture shown in red. These are there to help you see your matte, very useful especially in situations where you have lots of similar colors around moving objects. Above that is the Roto Brush Span. This lets you click-and-drag the settings you’ve made in one frame over a longer span of frames, saving you from having to make the same moves over and over. Case in point: My victim is in most of the beginning of the shot alone. I go one frame before my protagonist’s hand appears. Roto Brush has selected my victim. I hit ALT, leaving my cursor red. I make a stroke over his body, and he is removed from the matte.
Now instead of having to do that frame-by-frame, I go down to the Roto Brush Span, and find my edit, represented by a small yellow box. On either side of the yellow box are grey arrows. I can click-and-drag the arrows pointing to the beginning of the timeline, replicating that edit’s settings for the other frames that come before my edit. A green bar pops up in the middle of the Roto Brush Span to show what frames have been rendered.
This does requires tweaking, but the results and the time saved are worth it. Once my matte is finished, it sits over the original, with blood in the middle. My effects sandwich is complete. It now looks like blood is coming behind my protagonist, and out the victim’s body. After a little tweaking of the blood, I’m ready to render my finished shot. Let’s look at the final result:
Thanks to MPS Digital Studios and Hellevator!