Google says that it is testing a new feature on its YouTube Stories app that allows users to change video backgrounds in real time, without the need to use a green screen, or complicated software.
Known as “video segmentation,” the process of separating foreground from background in a video is notoriously difficult to do. If you’re not a pro, and/or not using a green screen. Doing it would need hours of work, if not a professional video setup. But Google says it is developing AI that will be able to do mobile real-time video segmentation on the fly – right within its beta Stories app. And all you will need is your smartphone.
The AI technique uses very much the same process that a human would undertake; it takes each frame and identifies and cuts out the background. The difference is that it gets done really fast. As this is done, the background can be replaced as well. How did Google make it work? Well, after being taught how to segment one image, the AI can make comparisons with the next and the next etc.
If there are small differences between several frames (i.e. someone is talking into the camera) the AI adjusts the mask only slightly. When there are major differences between frames, the AI creates a new mask.
Of course, Google had to teach the AI to do this, so a lot of human input was necessary to begin with. Yet, the result is pretty impressive, especially if you think that this process can be run by a smartphone. After adjusting for speed and quality, the final result is very smooth, allowing users to replace a background in real time, at 100 fps on an Apple iPhone 7, and at over 40fps on a Google Pixel 2.
The feature is being beta tested currently, and rolling out slowly, but that will allow Google to figure out how to improve it, and hopefully add new options; AR for example.
You might also like
More from Youtube
YouTube Tests A New Way To View Shorts On Its Platform
YouTube started testing a new way to watch Shorts, its TikTok-like short-form video experience.
9-Year Old Ryan Kaji Is The Highest-Paid YouTube Creator
Forbes has named 9-year-old Ryan Kaji as the highest-paid YouTube creator for the third consecutive year. He earned $29.5 million …
Tired Of Billie Eilish’s ‘Bad Guy’ Video? Here’s A Version That Never Ends
To celebrate the 1 billion views of Billie Eilish's 'Bad Guy', YouTube Music created an interactive 'infinite' version - 'Infinite …
YouTube Will Start Showing Ads On Non-YPP Channels, Won’t Pay Creators
YouTube changed its terms of service in the US to allow it to show ads on non-YPP channels and not …
YouTube Launches Audio Ads To Help Advertisers Reach Listeners
YouTube has announced audio ads, its first ad format designed to connect brands with podcast or music listeners.
YouTube Cancels Rewind 2020
2020 has been a terrible year and YouTube wants you to forget it. That's why there will be no YouTube …
YouTube Music Playlists Just Got An Upgrade
YouTube has announced five new improvements for YouTube Music playlists, making it easier for users to create, discover, and share …
David Blaine’s Latest Stunt Will Air Exclusively On YouTube
David Blaine announced Ascension, his first major stunt in a decade; an online-only event to take place exclusively on his …