We launched NeuralCam in 2019, making it the first iPhone camera app capable of shooting high quality night photos well before Night Mode was officially supported on the iPhone. Since then NeuralCam has become the most popular Night Mode camera app for the iPhone with millions of photos taken all over the world.
As time passed however, we discovered that people were using NeuralCam for a lot more than just night photography. It turns out that the technology we have developed to capture high quality night time photos is just as useful for daytime photos. The multi-frame merging reduces noise and the brightening AI model lights up the photo for a unique bright look that many people love especially in indoor, backlit or other high dynamic range situations.
By Robert Bilinsky, a professional photographer and NeuralCam user from London
(Robert Bilinski)
NeuralCam 5
So we decided to build a major update for NeuralCam, one that maintains its best-in-class night mode while adding some major improvements to make it the best daytime AI camera too.
Today we’re launching this new version, NeuralCam 5, and it’s the most advanced AI and computational photography camera for the iPhone, night & day.
NightMode: Still the best Night Camera
NeuralCam 5 maintains the same great night mode it’s known for and even improves it by optimizing on speed, adding full resolution support for all devices and 12-48MP AI Enhancement which further reduces noise and enhances detail in nighttime photos.
DayMode: Better daytime iPhone photos
But the biggest news for NeuralCam 5 is the features it brings in what we call DayMode. Actually the iPhone already takes some amazing photos using the default camera: Smart HDR and Deep Fusion do a great job in most lighting conditions. So, is it even possible to improve on this to further increase the quality of the photos? We thought about this a lot, did a lot of experiments, talked to NeuralCam users to see how they are using the app for daytime photos and one by one we reviewed all of the new AI technologies available that could improve daytime photos. In the end we came up with a new 4 step computational photography process and 4 corresponding technologies that when combined, improve iPhone photos in many different ways.
DayMode is actually not a separate setting, it’s more like a generic name for a multitude of settings that can result in better daytime photos. The easiest way to use DayMode is to leave every setting on auto and NeuralCam will automatically set the right capturing mode, number of frames and brightening. However, if you prefer a certain look or mode, you can use the computational photography selectors in the app to customize it.
Let’s see how this new NeuralCam AI Photography Process works from start to end.
The NeuralCam Computation Photography Process
To see what each of the technologies do, let's go through them one by one. In order to illustrate the different steps we’ll call for a bit of help from this Szekler wooden character, Pista Ba’:
SmartCAPTURE
First we’re going to start with capturing the best image possible. Just like the iPhone’s camera, the corresponding Camera APIs in the iPhone iOS SDK have become more and more sophisticated over time. The default camera’s UX stayed the same but at this point there are many different modes and formats that can be used to capture photos. From Burst-mode capture to Smart HDR, Deep Fusion or even the newest ProRAW, each of which has their own advantages and disadvantages. We’ve researched this quite in depth and translated all this into 3 different “Capturing Modes” that users can select: High Detail, Low Noise and Unprocessed. These do exactly what their names say, optimizing for either the highest amount of detail or the lowest amount of noise. In the case of the iPhone 12 and 13 Pro there is a third option - Unprocessed - which provides a minimally processed image keeping it very close to what comes straight from the camera sensor. In this mode there is no added sharpening, detail, or contrast boosting, just a beautiful soft image full with unprocessed information in every pixel. Things become even more interesting when, as we’ll see in the next step, we merge multiple of these high information images to create an even better one.
There is also an Auto option where the app will choose the right mode based on the lighting situation, trying to maintain the maximum detail while keeping the noise low in each case.
Looking at a close-up you can see how the optimized SmartCAPTURE in High Detail Mode provides a sharper, cleaner look, than the default camera.
Default iPhone camera app (left) vs SmartCAPTURE in High Detail Mode
SmartMERGE
In the next step we’re going to take multiple of these images and combine them with our SmartMERGE algorithm in order to reduce the noise and further increase the amount of information in the image, massively increasing the image’s signal-to-noise ratio. There is a reason we call this algorithm “smart”, and not just “merge”. SmartMERGE is much more sophisticated than a simple image averaging algorithm since it has to do advanced image processing in order to keep the image sharp and avoid blur caused by the movement of the camera or object movement in different parts of the image.
The result is a cleaner image with less noise and more information. The quality improvement is especially visible in areas with shadows and in high contrast scenes.
Default iPhone camera (left) vs SmartMERGE (right)
NeuralENHANCE
At this point we’ve got a high quality image that is high on detail and low on noise, which is a great input to our next step: enhancing or upscaling it with AI. Our specially trained AI Enhancement models enrich detail and sharpen the image at 12MP or upscale the image to 24MP or even 48MP. Also notice how our enhancement algorithm, in addition to enhancing detail, reduces the noise, and generally cleans up the image too. The end result being a much sharper and higher detail image with even less noise.
12MP Photo vs 48MP AI Enhancement Mode
We are particularly proud of this NeuralCam technology. This is state-of-the art AI super-resolution at work, something that until now was only available in specialized desktop editing apps like Lightroom or Topaz Gigapixel AI. Now thanks to massive optimizations we made we can bring this technology to the iPhone and it’s available to every NeuralCam 5 user on all iPhones all the way back to the iPhone 6s.
Our NeuralEHANCE iPhone-optimized Super-Resolution algorithm can output upscaled photos at a rate of 24 mega pixels per second on the iPhone 13 Pro, (and 12MP/s with a lightweight version on the iPhone 6s) which makes deep learning-based AI super-resolution feasible on the iPhone in a photo camera use-case. This makes NeuralCam the first (AI Enhanced) 48 megapixel camera for the iPhone.
NeuralHDR
For some time NeuralCam has provided multiple AI brightening modes for night mode photos. With NeuralCam 5 we’re introducing another brightening mode, especially tailored for daytime photos. It naturally brightens up the image and makes it pop. It’s especially helpful in backlit or high dynamic range situations (where the default iPhone camera doesn’t excel at), but after some testing we ended up using it for most of our daytime photos. On Auto mode NeuralCam will apply the NeuralHDR brightening in DayMode and a night mode specific brightening in Night Mode.
Default camera (left) vs NeuralHDR Brightening (right)
Better Photos Night & Day
So these computational photography steps, when combined, create a less noisy, sharper image at a higher resolution and with a lot more detail, which we say is a major improvement compared to the quality of the photos the default camera (or other third party camera apps for that matter) can provide.
When you are in a situation when you want the absolute best photo your iPhone can take, NeuralCam 5 is probably your best choice.
NeuralCam 5 - The most advanced computational photography camera
Besides this proprietary photography process, NeuralCam 5 provides additional features that make it the most technologically advanced camera for the iPhone.
Macro Mode on all iPhones
The new iPhone 13 Pro offers a dedicated Macro Mode, where the default camera automatically switches to the ultra-wide camera once it detects that the user intends to take a close-up photo. With NeuralCam 5 you can have Macro Mode on other iPhones too by simply selecting the new Macro Mode with one tap. By automatically selecting the right lense that can focus at the right distance and combining it with our NeuralENHANCE AI algorithm, we can get some really high quality macro shots, bringing a selectable Macro Mode to all iPhones.
And since this is NeuralCam’s Macro Mode, it’s also available in Night Mode and can be combined with 48MP and 24MP AI Enhancement modes too. So we can shoot 48MP Night Mode Macro photos, something that is only possible with NeuralCam.
Manual computational photography controls
Since Apple builds its iPhone camera for the average user and not for pro photographers, it tries to keep it as simple as possible, so the more dedicated iPhone photographers have always been searching for manual camera apps that allow them to customize and set the right camera settings for each scene. While there are multiple Pro / Manual Camera apps in the AppStore, NeuralCam 5 is the only pro-level manually customizable computational photography camera app out there. For the first time, photographers can choose between different capturing modes, number of frames and different AI enhancement options, and even combine them with classic camera controls (that are compatible with the computational photography process) such as white balance and exposure compensation.
UI Changes
We’ve also made several user interface changes in order to make the whole user experience faster and more suitable for daytime photography. These include removing the photo review screen, and moving all of the controls to the main screen and adding explainer popups to make it easier for new users to understand this new computational photography process and how to control it.
Available now
So this is NeuralCam 5, the most advanced computational photography camera for the iPhone. Give it a try, start experimenting with it, and let us know what you think. Whether you’ll use it in auto mode or fully customize the settings for each shot and lighting situation, we can’t wait to see what you can do with it!
It’s available with a 30% discount for a limited time! Please note that the 24MP and 48MP modes of the AI enhancement algorithm are available as a separate In-app purchase.
Get it at: https://neural.cam/nightmode