Needless to say, a video surveillance camera includes various components. Chief among them is the systems-on-chip or SoC. This article examines the role SoCs play and how they enable the camera to become smarter and more advanced.
Needless to say, a video surveillance camera includes various components. Chief among them is the
systems-on-chip or SoC. This article examines the role SoCs play and how they enable the camera to become smarter and more advanced.
Among the various components in a video surveillance camera, the SoC stands out and plays a critical role. “Regarding overall performance and picture quality, the elements within the camera that define image quality include the lens, image sensor and SoC. The SoC’s image processing is critical to the overall image quality, and the ability to do high-quality video compression also defines both the image quality and bitrate required for video streaming and storage,” said Chris Day, VP of Marketing and Business Development at Ambarella.
Making cameras smarter than ever
The SoC also plays a key role in accelerating the camera’s analytics capability. This is important as more and more so-called
AI cameras are becoming available in the market.
“Ultimately the SoC is not only responsible for how the camera performs, but enables advanced capabilities such as deep learning to become a reality,” said Stefan Lundberg, Senior Expert Engineer at Axis Communications. “For example, in the past motion detection technology would be used to alert a security team that something or someone was in a restricted area. With a sophisticated next-generation SoC, analytics can be applied to a scene to tell you exactly what object has triggered an alert – whether it be an animal, person or vehicle. As a result, security teams can make informed decisions as to whether to disregard or investigate further.”
To support complex,
deep learning-based algorithms, the SoC should co-work with an AI accelerator.
“Inference is key to modern AI-based analytics. This is where images flow through the neural network to form the final decision, for example, a detection or a classification,” Lundberg said. “An SoC must contain a hardware-accelerated deep learning processor, specific for this type of compute-intense task. This means that the system is able to perform this Inference in parallel with all other processing.”
“The camera SoC must integrate an AI processor to provide deep-learning-based analytics based on neural network processing. Ambarella’s CVflow AI engine provides this capability and delivers high performance AI processing at very low power consumption. The CVflow engine is in addition to the general purpose arm CPUs, which are also integrated into Ambarella’s SoCs for general purpose processing such as camera control tasks and running networks stacks,” Day said.
Sourcing vs. own development
Camera vendors get their SoCs primarily in two ways. They either source from major suppliers like Ambarella or use their own chipsets. Each method has its advantages.
“It is very expensive to develop SoCs, especially at the latest advanced process nodes, such as 5nm where a mask set alone (before considering VLSI engineering expense) costs multiple millions of dollars per design. Given the rapidly rising fixed costs to develop an SoC, a customer developing a captive solution will require more and more units to justify the investment. We are getting to the point where the costs have risen so much, fewer and fewer companies can justify the development of their own captive SoC,” Day said. “Additionally, most camera makers have a wide range of camera products that span different form factors, performance and features, making it impossible for a single SoC to efficiently address all these different camera requirements. Ambarella has multiple SoC families with different features and capabilities to address the full range of camera requirements, while sharing a common SDK for efficient software development.”
Meanwhile, some camera manufacturers like Axis use their own SoCs due to several benefits.
“When we produced the world’s first network camera – our Neteye 200 camera – we soon realized that available off-the-shelf SoCs could not deliver the level of performance we wanted going forward. These simply didn’t fit our vision for what we wanted to contribute to the surveillance industry and we dedicated many years to designing the SoC we needed,” Lundberg said. “This is the main advantage to designing our own SoC. The agility conferred means that we can design technologies in direct response to customer needs and a changing threat landscape. By owning the process end-to-end, any vulnerabilities can be identified and mitigated early in the process. This problem-led approach to innovation ultimately enables us to address their surveillance challenge, leading to better outcomes.”
Evaluating SoCs
When evaluating an SoC, there is a range of factors to consider. “It’s important to start with the basic capabilities. Once satisfied, you can then move on to the list of the most important SoC features. Next the supported standards must be right. It’s then critical to zoom out and ensure the whole solution is of good quality, consistently stable over time and only uses secure implementations,” Lundberg said.
According to Day, below are things the user should look at when evaluating SoCs:
- High-quality image processing, including advanced noise reduction and the ability to operate in challenging low-light conditions.
- High dynamic range processing to capture image details in high contrast scenes.
- Highly efficient video compression, using both H.264 and H.265.
- Dewarping of images, allowing wide angle views from fish eye lenses without image distortion.
- High resolution imaging and compression, providing more image details and quality over longer distances.
- Low power consumption, allowing the development of cameras with small form factors and longer battery life (in battery powered designs).
- Ability to simultaneously create multiple video streams with different resolutions and bitrates.
- Ability to connect to multiple image sensors in multi-imager designs, for example four sensors in a single 360-degree dome camera design.
- High performance AI processing for advanced analytics in the camera and the ability to run multiple neural networks simultaneously.