7. InM Component: Rotoscope any matte holes (reflections, etc) as a holdout matte and plug in here. Make sure to set to Alpha and if you’re in Nuke, set your Project Settings appropriately! Rotoscope nodes default to the Clip to Format, not the size of your keying footage!
8. OutM Component: Rotoscope garbage matte elements here (lights, rigs, uneven screen defects, etc). See the important roto notes in 7 above.
What about spill? Notice I didn’t touch the Alpha Bias to suppress green spill or contamination.
If you’re using Nuke, the HueCorrect is a much better choice. Sample the spill value, set your HueCorrection to g_sup (green suppression, not green), and bring your curve down to the 0 (transparent) range. You’ll find a much stronger suppression and will do little damage to other color ranges in your source footage.
Always shoot in a higher resolution than the final result will need, and as high a color depth as you can (16bit or HDRI). For low light or desaturated footage, do an initial color adjustment before you start to help the Keyer.
Don’t worry what your key is doing to your RGB pixel source values – you’re only concerned about generating a great matte channel. I always shuffle my new alpha back into the original RGB pipe after the key anyway (this will also serve to negate any temporary color correction used earlier.
And of course never leave a background plugged into the node and leave the View on Composite. That leaves no room for additional color correction/animation/compositing on the individual elements if you merge it back to another background right away! The final Keylight should be left at Final Result in the View dropdown.
For many more tips on green screen shooting and keying, see my other pages at Visual Effects > Green Screen Studio Tips!
Watch for the advanced Keylight information pages coming soon!