Adobe launches VideoGigaGAN: AI enhances blurry videos

Adobe Research's team of video and artificial intelligence engineers has developed an artificial intelligence application called VideoGigaGAN that can take blurry videos and enhance them to make them clearer and sharper. The team has published an article on the arXiv preprint server describing their work and results. They have also released multiple enhanced video examples on the project website.

Recently, artificial intelligence applications have been frequently featured in the news, mainly due to the release of large language models (LLMs) such as ChatGPT, which allow consumers to generate various outputs using these models. However, AI research is also ongoing in other areas, such as creating artificial images and videos.

In this new research, the Adobe team has created an application that can take blurry video samples and return the same samples with higher clarity and sharpness through processing - a process commonly known as upsampling.

This application is named VideoGigaGAN - a name derived from their previously showcased application, GigaGAN, which can generate new photos or enhance old ones. GAN stands for Generative Adversarial Network.

As the name suggests, the team uses a generative adversarial network to teach the system what constitutes a clear and sharp video (e.g., individual hair strands on eyebrows instead of a blurry mass) and then adds a "flow-guided propagation module" to maintain consistency between video frames.

They also employ anti-aliasing techniques to prevent what they call "AI artifacts" and high-frequency feature transfer techniques to address unexpected drops in video quality.

The team claims that their result is a system that can improve video image quality by up to eight times - all without introducing strange colors, uneven lines, or other well-known issues associated with AI-generated images and videos.

They acknowledge that some of the outputs are entirely artificially generated based on the system's estimation and are intended to fill in missing details in the images. For example, pores on the skin, lines around the eyes, and even eyelashes are added to make the generated videos clear.

The team notes that the current release of the system is only a demonstration and not an upcoming product release, so it is currently unclear whether Adobe will make it available to the public.