Google says that it is testing a new feature on its YouTube Stories app that allows users to change video backgrounds in real time, without the need to use a green screen, or complicated software.
Known as “video segmentation,” the process of separating foreground from background in a video is notoriously difficult to do. If you’re not a pro, and/or not using a green screen. Doing it would need hours of work, if not a professional video setup. But Google says it is developing AI that will be able to do mobile real-time video segmentation on the fly – right within its beta Stories app. And all you will need is your smartphone.
The AI technique uses very much the same process that a human would undertake; it takes each frame and identifies and cuts out the background. The difference is that it gets done really fast. As this is done, the background can be replaced as well. How did Google make it work? Well, after being taught how to segment one image, the AI can make comparisons with the next and the next etc.
If there are small differences between several frames (i.e. someone is talking into the camera) the AI adjusts the mask only slightly. When there are major differences between frames, the AI creates a new mask.
Of course, Google had to teach the AI to do this, so a lot of human input was necessary to begin with. Yet, the result is pretty impressive, especially if you think that this process can be run by a smartphone. After adjusting for speed and quality, the final result is very smooth, allowing users to replace a background in real time, at 100 fps on an Apple iPhone 7, and at over 40fps on a Google Pixel 2.
The feature is being beta tested currently, and rolling out slowly, but that will allow Google to figure out how to improve it, and hopefully add new options; AR for example.
You might also like
More from YouTube
Smaller Creators Can Now Access YouTube Partner Program
The YouTube Partner Program will now be open to creators once they reach 500 subscribers, allowing smaller creator to monetize …
YouTube Brings Unskippable 30-Second Ads To TV
YouTube is bringing longer unskippable ads to its TV app, unless you're paying for the platform's premium subscription YouTube says these …
YouTube Brings Shopping Features To Shorts
YouTube is testing affiliate marketing and in-app shopping features on Shorts, less than a week after TikTok began testing similar …
Get Ready To Choose Your YouTube Handle
YouTube is introducing handles to make it easier for members to find and connect with each other. Your handle will be …
YouTube Now Lets You Add A Voiceover To Your Shorts
YouTube is adding one of its most-requested features to Shorts; the ability to add a voiceover, like narration and original …
YouTube Starts Adding Watermarks To Downloaded Shorts Clips
YouTube has started adding watermarks to downloaded Shorts clips to stop users from cross-posting them to other platforms.
YouTube Expands Offense Warnings To Desktop, Updates Payments System
New updates for creators! A dedicated payments account within AdSense, and warnings on potentially offensive text detected by AI.
YouTube Offering Podcast Networks Up To $300k To Film Episodes
YouTube is offering financial incentives to podcasting networks to adapt their shows to a video format and share them on …
YouTube Adds Guided Support For Community Guideline Violations
YouTube has updated its review process to provide creators with specifics about the reportedly violative video and facilitate a resolution.
