Though smartphone cameras have improved a lot over the years, low-light photography still seems like a arduous task and the images can turn out grainy or blurred or just plain bad. Nvidia is working on a new tech that makes it possible to fix such grainy photos.
In an official blog post, Nvidia highlights its deep learning-based model that has learned to fix your grainy images simply by looking at other similar ‘corrupted’ photos. The AI works based just on other images captured in similar low light conditions, and you don’t need to feed it any clean images as a reference.
The researchers working on this project say that most AI models are usually trained with two sets of images — a pair of noisy or grainy photos and a pair of clean images. The AI learns to differentiate between the two and then use them to learn how to clean up your grainy photos.
However, Nvidia is leapfrogging the tradition AI model as its deep learning-based model requires just two images with noise and grain. This model has never been shown what a clean image looks like, but it is capable of producing a similar output, as said in Nvidia’s research paper.
The researchers working on this project used Nvidia Tesla P100 GPUs with a cuDNN-accelerated TensorFlow deep learning framework. This helped train the systems on over 50,000 images in the ImageNet validation set (which is the dataset used to train the AI models).
The researchers tested their neural network with 3 different datasets and concluded that it’s on par with other state-of-the-art methods that make use of clean examples – using the same training methodology. Talking about its breakthrough AI model, the researchers stated in the paper, “It is possible to learn to restore signals without ever observing clean ones, at performance sometimes exceeding training using clean exemplars.”
Nvidia believes that there are several real-world situations such as low-light photography, MRI or renders, where one would need to get clean training data. The system sure has its shortcomings you can see in the output but “all the proof-of-concept demonstrations point the way to significant potential benefits in these applications by removing the need for potentially strenuous collection of clean data.”
