Great mobile cameras are built from solid imaging hardware, traditional imaging algorithms, and increasingly important AI Engines to convert those digital 1s and 0s into great-looking snaps. The latter is advancing in leaps and bounds thanks to the growing power of machine learning and computational photography. AI technologies are helping smartphones produce professional-quality photos with beautiful bokeh, high dynamic range images and film, and even video capture in near darkness. But ever-improving software techniques also require more powerful machine learning and image processing hardware, such as those found in Qualcomm’s Snapdragon 888 5G mobile platform.
The Qualcomm Snapdragon 888 operates as a heterogeneous compute platform, distributing machine learning workloads to the most appropriate component. When it comes to photography and video, the platform’s Spectra 580 Image Signal Processor (ISP) and Hexagon 780 processor play the core roles.
Smart image processing that understands your pictures
The Snapdragon 888’s Spectra 580 ISP handles all the quintessential photography and video processing stages. It’s the core component that takes the image from the camera sensor and turns it into the digital 1s and 0s stored in your smartphone’s gallery. But not satisfied with image processing basics, Qualcomm augments its ISP with additional features designed to help users capture the perfect picture. Let’s dive into some examples.
The Spectra 580 is actually a “triple ISP” capable of processing a colossal 2.7 gigapixels per second. That’s enough data throughput to record 8K video or capture huge 200-megapixel images, which equates to more detail than ever before. But the triple ISP also allows the processor to capture pictures and video from up to three cameras at once, enabling picture-in-picture capture such as Director’s View in the Samsung Galaxy S21 series. The ISP also offers a full 10-bit image pipeline, allowing for full capture and storage of over one billion shades of color using the HEIC image format, as seen in the Oppo Find X3 Pro. That’s a lot more imaging data than the conventional 8-bit JPEG.
Qualcomm’s Zero Shutter Lag feature “pre-captures” images so that they’re ready to go as soon as you press the shutter button so you won’t miss that perfect fast-action shot. The ISP also processes multiple image frames for noise reduction and high dynamic range, ensuring that details look crisp and smooth even in tough lighting environments. No one wants to reshoot when their images don’t turn out right, and Qualcomm’s use of intelligent image processing techniques ensures this won’t be necessary.
But that’s not all. The Spectra 580 inside the Snapdragon 888 also boasts “Computer Vision” capabilities built directly into the silicon, imbuing it with AI imaging smarts.
Autofocus, auto-exposure, and auto-white balance are the imaging cornerstones that ensure your images look professional quality regardless of the scene you’re shooting. For example, you’ll want different exposures for landscape and portrait shots. By making use of AI smarts directly in the ISP chain, Qualcomm’s camera setup makes these important adjustments in real-time without taking up precious processing resources and battery life. But most importantly, it ensures your pictures will always be in focus, correctly exposed, and have the right looking colors every time you press the shutter button, whether you’re shooting a macro shot, portrait photo, or a beautiful landscape.
If you’re after some examples of just how powerful these technologies are, check out these pictures captured on Snapdragon devices.
AI smarts for a professional touch
While the Snapdragon 888’s ISP handles core image and video processing tasks, the Hexagon 780 processor inside the 6th gen Qualcomm AI Engine stands ready to step in when additional machine learning horse-power is required. Use cases involving the Hexagon 780 processor can be far-ranging, from sharpening up the look of your pictures to completely overhauling a smartphone’s imaging capabilities.
Software bokeh blur is a hugely popular photography and videography feature and Qualcomm leverages its AI Engine to produce high-end results. Hardware-accelerated machine learning calculates foreground and background depth and edges, while a graphics pass applies the software blur to the image. The result is images that take on a naturally soft look often associated with premium DSLR cameras. Qualcomm’s AI Engine is even fast enough to run all this math in real-time to apply bokeh blur to 4K 30fps video footage.
For another example, ArcSoft developed an AI Autozoom feature that’s accelerated on Qualcomm’s AI Engine to automatically follow and zoom on objects and people in your scene. This feature, which is included in the Smartphone for Snapdragon Insiders, is applied in real-time, allowing you to switch subjects, and automatically zooms in and out and even switches to the most appropriate camera lens in a multi-camera smartphone setup. You’ll never lose track of fast-moving objects and subjects while shooting video ever again.
Qualcomm also partners with BlinkAI, which leverages the Snapdragon 888’s AI Engine for vastly improved night video shooting with the Xiaomi Mi 11 smartphone. BlinkAI’s computational solution improves effective illumination by up to 5x to vastly improve image quality in tricky shooting environments. So long dark, gloomy, and grainy video footage when shooting in low light.
Powerful hardware allows computational video capture to go even further, enabling multi-frame imaging techniques we usually associate with photography to now be applied to video. For instance, the use of Digital Overlap (DOL) and Staggered HDR image sensors combined with processing smarts merges multi-frame together for powerful HDR video, even at high resolutions. This technology is found inside the OnePlus 9 Pro, Oppo Find X3 Pro, and Xiaomi Mi 11 Ultra.
Computational photography is the future
Along with improved camera lenses and sensor hardware, cutting-edge AI Engine capabilities are essential for driving smartphone photography and videography innovation forward. But your handset needs the necessary hardware to make this a reality.
The combination of Spectra 580 triple ISP and Hexagon 780 AI-focused processor inside the Qualcomm Snapdragon 888 provides the capabilities needed for next-generation mobile photography. The ISP provides the necessary data throughput for 4K video and multi-camera technologies that now permeate the smartphone industry. While the Hexagon 780 provides additional processing power for the most advanced computational photography techniques around, including object segmentation and real-time bokeh blur. As well as improving the consistency and quality of your photos, Qualcomm’s AI Engine is driving entirely new ways to capture the perfect snap and video too.
Importantly, AI camera features are creating major differentiators between products, giving users more powerful tools to not only capture the perfect shot every time but also to become more creative with their photography and videography. Whether that’s emulating the soft depth of field from a DSLR camera, accurately tracking your subject, or replacing entire backgrounds in real-time, powerful AI Engines are driving mobile photography to new heights.