Computational photography is one of the greatest gifts (and therefore curses) technology has brought us. Terminals like the first generation Google Pixel showed: the phone can see more of usespecially in night photography.

But shooting in low light is a big challenge. The sensor will produce more noise than necessary if the brightness is low and two decisions can be made here: respect noise to preserve detail or remove noise (delete photo) so you lose detail. This last decision is most popular on smartphones, and Google is already training an AI to solve this problem.

Google wants noise-free, but detailed photos

MultiNerf is Google project released as open source to solve night photography problem. Users still have a certain aversion to digital noise and Google algorithms want to fix the problem without uploading the details of the photos along the way.

Google relies on NeRF (Neural Radiance Fields), a neural network developed in the first place to create 3D images based on 2D clusters. The point of building on NeRF is that, AI from the 3D image makes it easier to analyze the information of the image because it can “move” through it.

“We modified the NeRF to train directly on linear raw images while preserving the full dynamic range of the scene. By creating raw output images from the resulting NeRF, we can perform new high dynamic range (HDR) view synthesis tasks. From the camera’s perspective, focus after event, exposure and we can change the tone mapping.”

The algorithms of this model analyze the raw data of the RAW file and use artificial intelligence to detect what the information would be if there were no noise in that scene. In an other saying, they try to preserve the details of the RAW even if the noise is removed later.

While this model is still being researched, it lays an important foundation for the future of computational photography. Let’s hope Pixels start implementing this technology before long.and that it has spread to the rest of the manufacturers.