Many types of video analytics can be performed either at the server or at the edge (on camera). Performing video content analytics at the edge helps conserve overall processing capacity since the video analytics is done before the image is compressed. This means that the image will not need to be decompressed again at the server for video content analysis.
Many types of video analytics can be performed either at the server or at the edge (on camera). Performing video content analytics at the edge helps conserve overall processing capacity since the video analytics is done before the image is compressed. This means that the image will not need to be decompressed again at the server for video content analysis. Instead, metadata (data about data) provides the necessary information for easy identification and retrieval. A truly future-proof solution is one that enables some part of video analytics at both points server and edge. This is partly because some types of video content analysis will always require the greater processing power and speed of a central server and also because we will see more and more edge products with the capability of performing some video content analysis. As this happens, the need for a strong video management system with an analytics framework will become even more evident as the value of querying across metadata from different edge devices from different vendors will be very high.
On-camera analytics
Performing video analytics at the edge can significantly reduce bandwidth demands. You can set up your system to store only video identified as being ¨of interest〃 such as clips indicating a security breach, unauthorized entry or fulfilling other criteria. In this case, you manage by exception to normal events, collecting what the on-camera analytics are designed to recognize. On-camera analytics can be a solution to two of the biggest deterrents to the widespread adoption of megapixel (high resolution) cameras: bandwidth and storage. In fact, the two work hand in hand. Megapixel cameras provide the detail necessary for many video analytics products to perform complex operations. On-camera video analytics can make sure you only disseminate through the network and store video of interest.
Todayˇs on-camera analytics are capable of processing images in real-time and encoding archived video with metadata in MPEG or other formats. This metadata enables indexing of the relevant image content for fast retrieval. In fact, adding meta tags for user-defined events allows retrieval within seconds of specific events from hours of video from thousands of cameras.
The latest generation of IP network cameras combine recent chip design advancements with better compression technologies (such as H.264) to enable high definition with less bandwidth. This further paves the way for large-scale surveillance systems using megapixel cameras and on-camera analytics. In addition, advanced digital signal processing (DSP) technologies bring even better on-camera video analytics within reach. Hardware accelerators do the heavy lifting of encode/decode and display, enabling the DSP to handle the intense processing requirements of video analytics. Several DSP makers have teamed with different video analytics developers to release camera reference designs, and there are several standardization attempts in the marketplace today. Standard reference designs and recent efforts to put analytics on board cameras using one chip instead of two or more promise to lower the price per channel over time.
Server-based analytics
On-camera analytics makes sense for the images coming from one particular 
camera. When you need to perform analytics or correlate metadata on live or stored video from hundreds of cameras, you need the greater processing power and central management capabilities of a server-based solution. Server-based analytics enable more complex analytics and fast searches through archived video. Server solutions are critical for making a variety of video analytics tools available to various departments, from security to HR, operations and marketing, and allowing searches for after-the-fact criteria or analytics (such as facial recognition or object identification) that are beyond the capabilities of on-camera solutions.
A relatively new solution, image processing over IP (IPoIP) technology, distributes the image processing responsibilities between camera and server. The video analytics algorithms are segmented into two parts and divided between the video encoder hardware and the central image-processing server. IPoIP retains the strengths of both distributed on-camera and server architectures, while avoiding their limitations.
How should you set up your system? This ultimately comes down to your needs, as well as the IP video surveillance management solution you choose, the cameras, the video analytics solutions, and the various hardware components. A qualified solution provider will be able to help you make the right choices.
How Milestone is helping accelerate the implementation of video analytics
Milestone is working on a Video Analytics Framework that will collect metadata from video analytics products (server-based and on-camera) in a standard database and correlate them in real-time or afterwards. Milestoneˇs query engine provides fast analytic searches of the metadata, allowing organizations to find video of interest fitting a number of criteria or particular events.
A combination of advanced analytic tools and optimized IP-based video technology based on the Milestone open platform, XProtect Analytics Framework enables video analytics at the edge (on camera) and at the server, minimizing processing power and adding value to archived video. With XProtect Analytics users can correlate events from generic tools such as license plate recognition, facial recognition and traditional real-time access control with alerts from video content analysis tools, such as object detection, etc. Users can build strong, accurate evidence by cross-matching events in real-time and from archived video.
Integrating video analytics with open platform video management systems gives a flexible choice of hardware and software that greatly expands the potential for video analytics and increases the value of archived video. A true open platform solution, such as the Milestone Video Analytics 2.0 Framework, simplifies system operation by integrating a wide variety of video analytics products under one easy-to-manage user interface. This gives a powerful surveillance solution with a flexible IP video management system and a central console for operating, collecting and correlating events from multiple sources.
The shape of things to come
One thing the industry must do is keep driving towards open platform solutions. The security industry has long been one full of proprietary systems. This needs to change and is changing rapidly with IP video solutions. Most customers want their video analytics solutions to be seamless and interoperable with other security devices and operate within their video surveillance management software. The key to this is open architecture and standards-based solutions. Applications need to be building blocks, not solitary solutions. Creating open solutions and developing standards for the industry will make it easier and more lucrative for developers to create video analytics applications for various markets.
Three of the largest IP camera makers Bosch Security Systems, Sony and Axis Communications have teamed up to work on a standard for the interface of network video products. It is expected to comprise interfaces for specifications, such as video streaming, device discovery, intelligence metadata, and other IP-surveillance integration. Products incorporating the standard could be available as early as the second half of 2009. Other companies are joining their forum. Another group, the Physical Security Interoperability Alliance is supporting the interoperability of all devices in the physical security industry. The National Retail Foundation is also working on an analytics standard.
Exciting new developments will drive new increased interest in video analytics in the years ahead. For instance ¨click and search〃 functionality is beginning to appear. This technology enables you to click on an object, such as a package, in an image and then search for all instances of that object in your stored video from all your cameras.
Gadi Talmon, Co-Founder of Agent Vi, a developer of enterprise video analytics software solutions, predicts that ¨in 10 years every video surveillance camera on earth will have some kind of intelligence. Some of them will have basic capabilities like motion detection and some others will have very sophisticated capabilities like recognizing suspect faces in crowded environments and automatically tracking them when moving between cameras.〃
These are exciting times. Itˇs not too early (or too late) to start using video analytics in your business or organization. In fact, the time is ripe for letting your video surveillance system do the watching.
Product Adopted:Software