A team of researchers at Nvidia led by Guilin Liu has developed a new technique which makes use of artificial intelligence (AI) to repair corrupt or damaged images. According to an official blog post on the matter, the technique users a state-of-the-art deep learning method that can edit images or reconstruct a corrupt image which has holes or missing pixels.
In addition to repairing images, the method can also be used to edit the images by removing content and filling in the resulting holes. In theory, the method performs a process called “image inpainting” and can be implemented within a photo editing software to remove unwanted portions of images and filling them in with a realistic computer-generated alternative.
The team has submitted a research paper on the technique which states:
“Our model can robustly handle holes of any shape, size, location, or distance from the image borders. Previous deep learning approaches have focused on rectangular regions located around the center of the image, and often rely on expensive post-processing…Further, our model gracefully handles holes of increasing size.”
The researchers trained the neural network by first generating 55,116 masks of random streaks and holes of arbitrary shapes and sizes. The team also generated 25,000 more such holes for testing, which were divided into six categories based on their size relative to the input image. The team then made use of Nvidia Tesla V100 GPUs and the cuDNN-accelerated PyTorch deep learning framework to train the neural network by applying the generated masks to images from ImageNet, Places2 and CelebA-HQ datasets.
You can check out Nvidia’s inpainting in action in the video embedded above which displays rather remarkable results. While the end output is not always perfect, it’s still a remarkable achievement and shows a lot of promise. Of course with most machine learning technologies, this is likely to improve over time with more iterations and a bigger pool of data. Additionally, the same framework can be used to handle image super-resolution tasks, making it even more useful.
