Hello NeuralCam fans,
A year has passed since the launch of our NeuralCam Live iPhone app, and we’ve got a new product to show you today that similarly aims to improve video calls but by taking a slightly different approach.
Laptop webcams
As we were forced to turn our living room into a home office our expectations around the quality and overall user experience of video conferencing have increased too. Much of the time we use our laptops to do video calls and each time we do, we’re reminded just how far behind laptop webcams are in terms of image quality. It just doesn’t cut it anymore.
External Webcams & Smartphones As Webcam
In order to get around this laptop webcam quality limitation, people started trying different solutions like buying external webcams or using applications that turn smartphones into webcams. We launched NeuralCam Live for iPhone last year, that does exactly that, and more.
Both of these options provide a major upgrade in terms of video quality, but none of them is ideal: webcams can cost a lot of money and act as yet another device we need to carry around and set up for every meeting.
Phones used as webcams are a step ahead, because we always have them around and they provide exceptional image quality, far better actually than the average external webcam. However, phones also need to be set up for every meeting in the same way external webcams do, a separate stand is required which often leaves us scrambling to set things up minutes before an important meeting. These can be the right choice for rare, special meetings, but for the majority of video calls we need something simpler and easier to use.
NeuralCam Live for Mac
We wished we just had a better built-in webcam - something better in terms of image quality, but no more complicated to use than the default laptop webcam - and this is how NeuralCam Live for Mac was born. It is a virtual camera app, just like NeuralCam Live for iPhone, but unlike the iOS version it doesn’t need any additional hardware to work, instead it aims to upgrade the quality of the Macbook’s built-in webcam using advanced machine learning algorithms.
The new Apple Silicon powered Macbooks are redefining the concept of laptops and what they can do, massively increasing computational power while at the same time also using less energy. Especially interesting for us as a company that builds smart camera apps is that the M1 chip also includes a powerful NeuralEngine which makes Apple Silicon Macs the first mainstream AI computing platform capable of running multiple AI models on an HD video feed in real time for a sustained period of time. This new capability opens up the way for entirely new types of applications, NeuralCam Live for Mac being one such app.
The Macbook Webcam
First, let’s mention that the M1 Macbook’s webcam is a significant improvement in itself over previous Mac webcams. While the camera hardware and resolution didn’t change, Apple managed to get a visible quality improvement by updating the ISP (Image Signal Processor) that offers better noise reduction, auto-white balance and a greater dynamic range.
Let’s take a look at two video frames coming from the M1 Macbook Pro’s FaceTime HD camera:
The first one was shot in fairly good light, probably more than you get in a typical video call. Generally speaking the image looks good, but compared to a smartphone camera that we’re used to, it looks quite low resolution, soft and slightly blurry. It also lacks any kind of depth of field effect you would expect.
The second one is a frame shot in low light, at the darker end of the range when it comes to video calls. In this case the image looks much worse: it’s too blurry, and there is big grainy noise everywhere.
We want to improve the image in both of these cases, and we’re going to use multiple AI powered video processing algorithms to do so.
AI Enhancement
When there is good light and the image looks OK but blurry, we’ll use an AI powered image enhancement algorithm that applies some smart sharpening and adds a bit of micro-contrast to make the image look much more crisp and clean. It looks as if you upgraded your 720p camera to a higher quality 1080p one.
Background blur
Next we’ll apply another AI algorithm that beautifully blurs the background of the subject. By using the power of the M1 chip we can apply this algorithm more exactly and to every frame of the video, creating a smoother, more professional looking depth of field effect when compared to typical background blur implementations in video call apps. With both the AI Enhancement and Background Blur applied, the image already looks a lot sharper and generally of higher perceptual quality. But we have one more step to add.
AI Color Grading
Lastly, we’ll do some AI powered color grading. The Macbook’s webcam provides a pleasant, nicely colored video feed. However, in some cases and lighting situations, a different type of look might be better. A third Machine Learning model, that was trained on professionally color graded videos, applies a complex style to the image. It’s up to you to choose the best according to your situation or personal preference. For example the Clean style does a great job in using color contrast to isolate the background and draws the eyes towards the face.
Putting it all together
So, running all three AI models on the image - yes, the M1 is capable of running three optimized Machine Learning models in real-time on a HD video - we get this result:
Low light conditions are more challenging, but we only need one additional step to make our enhancement pipeline work in this scenario too.
AI Denoising
Once the amount of light starts to drop the noise becomes visible very fast and the overall quality of the video deteriorates a lot. Brightening that video frame would make it just worse with all of the noise becoming even more visible. Our AI powered denoising algorithm, especially trained to handle this type of noise, does a great job at removing noise while keeping the image sharp.
After removing the noise it is now possible to brighten the image and achieve a much more pleasant low light video.
The app
Install and forget
The app itself (just like the iPhone version of NeuralCam Live), is a virtual camera app that once installed and selected as the camera of choice in the video conferencing app, it will start automatically every time the video call app starts, providing the enhanced video with no additional effort required.
This automatic functionality makes NeuralCam Live the simplest video quality upgrade for Macbook webcams. It is also a great way to have the same consistent look and feel across all your video call apps.
Because it acts as a virtual camera all this functionality can be combined with the built-in functionality of the video call apps. For example you could use the AI Enhance and Styles in NeuralCam Live and add some specific background using Zoom.
UI and features
The UI of the app is simple too, containing a few buttons that allow the customization and combination of the features mentioned above: AI Enhance, Low Light Mode, Bokeh and Styles are each individually selectable and most of the features can be combined with each other so you can set up the right look for your environment or mood.
In addition to these features the app has a Head Bubble feature which uses face tracking to place the head in a soft circle, removing the background while maintaining a natural transition around the head. This mode can also be combined with other features and most of the time this is our preferred method for removing the background.
Price
NeuralCam Live for Mac has a free version that contains the two base styles and the Head Bubble feature, with all other features being part of the Pro version that costs $4.99/month.
NeuralCam Live for Mac is free for users who already have a NeuralCam Live for iOS subscription. However, since we had to use different payment methods for the two apps they are not automatically integrated yet, so users who are subscribers on one of the platforms and would like to use both versions of the app should send an email to support@neural.cam in order to get access to Plus versions of both apps.
Upgrading your M1 Mabook’s Webcam with AI
NeuralCam Live provides the easiest way to upgrade the quality of the M1 Macbook’s webcam with no additional hardware needed. Install it, configure the app one time and enjoy high quality professional-looking video calls after that with no additional effort.
It is available for download today, get it here: https://neural.cam/live/mac