While you can use smartphones in horizontal orientations, apps like Instagram, TikTok, Snapchat, and others highly encourage vertical video formats. As such, content creators and regular folks shooting horizontally have to crop their clips to fit these social media platforms — a tedious process. Adobe and other companies have been working on automating this process for years, but now Google is also entering the intelligent auto-cropping game with its open-source AutoFlip framework.
With AutoFlip, Google wants to provide a solution that makes reframing horizontal videos a hands-off process. The framework uses machine learning to recognize faces and key action in scenes and can spot cuts between shots. With this information, the algorithm intelligently crops and reframes clips. That also works for wild pans and follow shots, such as recordings of people skiing or pet videos.
Once AutoFlip is available publicly, it will also offer different configurations depending on your needs. You can require it to keep all detected faces visible in a shot, which the algorithm can ensure by adding padding to the top and bottom of clips. With the technology, it will be easy for content creators to shoot and edit one single video that can be output in any possible format with little to no further adjustments.
Google is currently working on finishing the algorithm internally, but it encourages contributions from outside developers and filmmakers as it's an open-source initiative. While the project might not hurt Adobe too much, it could end up being a tough competitor for smaller, specialized companies that offer automated cropping workflows in exchange for money — depending on how well Google's solution works, that is. I also wouldn't be surprised to see the tool pop up in Google Photos at some point.