This year the Secutech Excellence Award focused on exhibiting the performance of each product in specific applications and testing how the specs affected performance.
One of the highlights at Secutech 2014 was the Secutech Excellence Award, which was held for the fourth consecutive year. The shootout showcased two product categories: network cameras and NVRs. Participants included major brands from Germany, Korea, Japan, the U.S. and more.
Different from the approach we took in 2013, which was a comprehensive review of performance and technical details, this year we focused on exhibiting the performance of each product in specific applications and testing how the specs affected performance.
The products that stood out the most were the ones that were tuned onsite by manufacturers, which shows that calibration is crucial for any product to reach its full potential. In a highly competitive and mature market, hardware specs merely serve as an enabler—not a guarantee—of better performance.
Below are the observations we made at the 2014 Secutech Excellence Award.
Ultra HD a natural progression for network cameras
Manufacturers have focused heavily on developing network cameras that capture finer images and support higher resolutions. The resolutions of cameras showcased at the event ranged from 1.3 megapixels to 8 megapixels, showing that camera makers value practical performance over mindless pursuit of megapixel count when it comes to choosing an appropriate CMOS sensor. Dominating CMOS sensors at the event were Aptina sensors and Sony's Exmor and Exmor R sensors.
The deciding factor for image quality and resolution is the image sensor that manufacturers choose to incorporate into a camera. In our tests, we found that sensors with high megapixel count and high frame rate, such as IMX-124 and IMX-178, start at 3MP and go all the way up to 4K (800 MP). At the same frame rates, cameras that use these sensors showed better image quality and more accurate color reproduction than ones that used smaller sensors.
However, sensors with lower resolutions but high frame rates also performed well. Although some lenses introduce a certain level of distortion, all cameras supported multiple aspect rations, including 4:3, 5:4 and 16:9.
All 22 cameras that entered the shootout were capable of capturing HD video. The evolution of cameras will remain focused on better image quality and higher resolutions, and it is certain that we will see camera and image sensor manufacturers continue to push the limits in this area.
True WDR vs. high frames rates
The shootout was staged to simulate real-world circumstances in which the performance of different cameras could be clearly compared. Calibrating the cameras proved to be challenging for the engineers onsite, since the lighting was not only specifically set to test WDR capabilities—there were also various moving objects in the scene.
As mentioned in the previous section, CMOS sensors have improved greatly over the years, and show no signs of slowing down. Camera manufacturers are taking advantage of newer chips to provide higher frame rates like 60 fps or even 120 fps, which help capture clearer and more detailed images while reducing motion blur and ghosting.
However, in practice, there are some technical limitations. Not every codec is capable of processing high frame rates or other advanced features like noise reduction or 4-frame WDR. In other words, a camera that supports higher resolutions and high frame rates is not necessarily capable of 4-frame WDR or 2D/3D noise reduction.
It is clearly a challenge to incorporate all these features into a single camera. Camera makers need to choose the right image sensor, a suitable codec and a powerful CPU to reach this level of performance. Otherwise, manufacturers can only pick and choose between WDR, high frame rates and 2D/3D noise reduction, depending on which applications the camera is targeting.
The truth about low light performance and night vision
A key focus for this year's Secutech Excellence Award was low-light performance. The simulated environment was not designed to test the speed or sensitivity of the IR-Cut in “day and night” models, or even the effectiveness of IR LEDs in a 0 lux environment—in our opinion, these do not represent true low-light performance, and do not reflect the craftsmanship of camera manufacturers. We wanted to see how the cameras differed in the calibration of light compensation, contrast and other parameters.
Prior to the event, we tested the cameras' performance in varying levels of luminance: 20 lux, 7 lux, 5 lux, 3 lux, 2 lux, 1 lux and 0.5 lux. The cameras were set to DNX Max on with maximum light compensation; contrast, brightness and shutter speed were all set to the default settings or automatic mode.
At 1 lux, the images captured by most cameras were filled with white noise. Not only was color aberration a big problem, the images were also out of focus. Some cameras even failed to produce clear, colored images at 3 lux. While this may be because some cameras were not equipped with IR aspherical lenses, it does not change the fact that few cameras could perform well at 0.5 lux and lower. Image sensors were the primary factor for improved low-light performance, but most cameras failed to build on that advantage with better shutter performance and frame accumulation.
At the event, when the network cameras were placed into the simulated environment onsite, some cameras that did not perform well during our tests even failed to produce images at 0.5 lux and lower. This was caused by a more complicated scene, which the manufacturers' engineers overcame by setting image sharpening to higher levels. This resulted in overexposure when the simulated environment's lighting was at full brightness, and excessive contrast when it was darker. A real-life situation of similar circumstances would be a camera installed in an alley: images would be normal during the day, but when it gets dark and a street vendor turns on some lights and opens for business—cameras that are not calibrated properly will run into problems.
Fragmented standards persist in the network camera market
When it comes to interoperability and standardization, camera manufacturers either go with ONVIF or PSIA, or at the very least adopt handshaking protocols like UPnP. Of the 22 cameras that entered the shootout, some have already begun to offer China's GB-28181 as an alternative to ONVIF, which means that major brands around the world are beginning to focus more on China's highly competitive security market.
More information about Top 9 must-see Network Cameras
More information about Top 6 must-see NVRs