The green screen, which has a long history in both film and video, has gotten an upgrade thanks to Netflix and a new technique that combines the use of machine learning and requires lighting the actors in magenta. From the looks of it, the results are great. Until now, the easiest method of removing or replacing the background was with chroma keying, which requires placing actors against a brightly colored background and then having the camera filter out that color. Originally, the screen was blue and then later green because neither color matches any natural colors of skin tone or hair color.
However, the traditional green screen method has its downsides, especially problems with finer details like flowing hair, transparent objects, or anything else with a similar color to the background. This means that often times additional cleanup is needed in post-production to remove any colored glow around the object and any other edits to ensure the finer details appear in the finished product.
The new method invented by Netflix, which makes sense since they do a lot of filming of original content, has shown impressive results. The actor is still recorded on a green background, but they light them with red and blue foreground lighting (magenta). By filming with this configuration, the green channel shows the actor silhouetted against a bright, even background, which is used directly as a holdout matte, which serves as the inverse of the actor’s alpha channel.
Using machine learning they then restore the green channel of the foreground. According to the abstract of a paper called “Magenta Green Screen: Spectrally Multiplexed Alpha Matting with Deep Colorization”
“We demonstrate that our technique yields high-quality compositing results when implemented on a modern LED virtual production stage. The high-quality alpha channel data obtainable with our technique can provide significantly higher quality training data for natural image matting algorithms to support future ML matting research.”
One downside pointed out by Gizmodo is that having everything on set tinted inmagenta can make it more difficult to visualize and preview footage. Another problem they note is that the technique doesn’t yet work in real-time – so don’t expect to see your local weathercaster using it just yet.
Speaking to the New Scientist, Video producer Drew Lahat said, “It may work well in a fully controlled space, but it would have to compete with other new techniques like virtual production stages, and win over producers in real-world scenarios.”
