Your Business's Best Friend
Editor / Provider: a&s International | Updated: 6/29/2011 | Article type: Tech Corner
Anyone who hasever taken notes on a regular basis understands how limiting handwritten notes can be; finding specific information in a notebook can be time-consuming and frustrating. In the past, video archives were nothing more than a collection of footage waiting to be reviewed. With the advent of intelligent surveillance systems, raw video has been transformed into actionable information.
Video content analysis (VCA) has been plagued by bad business practices and unrealistic expectations, but the technology is, in many ways, still evolving and there have already been tried and tested real-life success stories that benefit from it.
The key for VCA to provide maximum value is having a clearly defined problem and realistic expectations of a solution. “There have been successful projects installed all across Europe,” said Gerard Otterspeer, CCTV Product Marketing Manager, Bosch Security Systems. “In Greece, for example, they use video analytics in tunnels to detect if someone is driving the wrong way or at the wrong speed.
That's directional detection coupled with speed detection. If a car stops in the tunnel for a certain amount of time, which is object loitering, the local authorities can be notified. Furthermore, a clear distinction is made between size and object shape, so the system knows if it's looking at a person or a car.”
Quality Comes With a Tag
What exactly is intelligent video? It is a system that analyzes millions of pixels at blazing speeds, Otterspeer said. “No matter what detection you choose, as simple as it may seem, all of it depends on the underlying algorithm. The first step is to analyze all those pixels and then identify objects. Only when you have the objects can you set the rules. So, it all starts with the quality of the algorithm, and this varies greatly from provider to provider.”
For example , in a complex environment where there are objects walking past one another in the background and foreground, the object IDs may be merging and splitting constantly since the camera is not aware of depth in the scene. “A lot of time and efforts have gone into optimizing ID tracking to ensure there is a right balance. If this is not done properly and two objects merge into one, it messes up the detection,” Otterspeer explained.
Many camera companies are giving away their software at low or even zero costs. However, you get what you pay for, which is nothing, cautioned Ivy Li, cofounder and MD of iOmniscient. “System integrators sometimes complain that the products they were using did not work — if they bought them based on price rather than quality and functionality, they should not be surprised.” Developing VCA algorithms requires a tremendous amount of time and resources. When a provider is giving them away for free, it is possible that they did not put in that much effort to begin with. Came r a s , DVRs a n d o t h e r hardware devices are becoming commodities with reasonable quality, leaving price the only thing to compete on, Li said. “This is not yet true for video analytics. The huge difference in quality drives the market and solution prices.”
Most companies already offer reliable, basic solutions such as line crossing, zone entry/exit and tampering detection. Price per channel for these is dropping since most companies can deliver, said Thejaswi Bharadwaj, Head of Civilian Technologies at Delopt. “However, for analytics that involve significant R&D and intellectual property such as people counting, PTZ tracking and ALPR, the prices will stay steady for quite a while.” Today, many camera and DVR manufacturers give away their software for free to sell their other products, Li said. “If you want a good, working product, you would have to pay for it. It's the same with transportation. You can get a bicycle very cheaply. However, it will not get you from Beijing to Paris quickly.”
"Many VCA solutions on the market today use video motion detection. Today's VCA uses advanced technology which applies machine vision to video scenarios in security and business intelligence applications," said Ed Troha, MD of Global Marketing at ObjectVideo. "Video motion detection is often used in products as an added component to drive hardware sales. These are not truly intelligent analytics and can have limited reliability."
The algorithms determine how intelligent the VCA is, said Patrick Lim, Director of Sales and Marketing for Ademco Far East. “There are products that use entrylevel processors but perform very intelligent functions without a problem. However, having a more powerful processor gives you some space to accommodate future firmware upgrades that can bring more complex algorithms and smarter features.”
VCA algorithms are very computationally intensive, and any increase in available processing power results in the ability to deploy more accurate algorithms without increasing solution prices, said Zvika Ashani, CTO of Agent Video Intelligence. “Computerized vision is a science that has many applications other than security, and advances are slowly applied to VCA, resulting in an increase in solution accuracy.”
Organizations that have the luxury of applying their algorithms to different industries have the greatest advantage since they can adapt and apply their algorithms to different applications, achieving economy of scale and knowledge-sharing among different projects, Otterspeer added. "For example, algorithm R&D could be centralized and later utilized across divisions such as automotive and security."
Different vendors take different approaches, but the underlying concept is the same. Classifying different rules based on accuracy helps. According to Bharadwaj, there are three categories:
1. Moving objects: people counting, line crossing, zone entry/exit;
2. Static/semi-static objects: object removal, unattended object, dwell time detection (not based on face detection), loitering, crowd counting, object classification; and
3. Special rules: camera tampering, PTZ tracking of moving/stationary objects, ALPR, face detection.
Solutions for the first and third categories are mature, Bharadwaj said. “With the second, we believe more R&D investment is needed for a foolproof solution that works in all situations to surface.” However, Li feels that the question is not about whether an application is ready, but rather if a particular supplier has a product that can work robustly in different environments.
There are four VCA rules that most users are familiar with, namely people counting, directional detection, camera tampering detection and object removal. “The concept can be easily communicated to customers, and the benefits are easily understood,” said Jukka Riivari, CEO of Mirasys.
The largest marketfor people counting is retail, Bharadwaj said. “Presently, most retail locations do not have a way to measure footfall traffic and correlate it with sales numbers. A real-life scenario is one in which a store has counting systems installed at all entrances, aisles and exits. These systems can provide various statistics about flow rate, occupancy and other information that can help optimize operations.” In terms of operational efficiency, people counting helps stores maximize staffing levels at both peak- and low-traffic periods, said Steve Gorski, GM for the Americas, Mobotix. “Hospitality is another vertical that also finds significant value through the use of VCA.”
The traditional method for retail counting was highly inaccurate and costly, requiring extensive wiring and many sensors, Lim added. “New top-down intelligent video devices come fully packaged. For chain stores looking to collect data centrally, IP network connectivity also saves a lot of money. Crowd counting for traffic flow in shopping malls is also a growing trend.”
Accurate people counting requires good processing power. “More often than not, multiple cameras are needed for a wide door/passage way. These cameras need to function as a single integrated sensing entity and generate a single count. It is also important to handle overlap between camera views to avoid over/under counting,” Bharadwaj said.
People counting is very valuable for indoor usage, especially in conjunction with access control. However, the greatest potential lies in commercial and retail settings where people want to know how many people are standing in a certain aisle at a certain time. For security and safety applications, there is increased use of crowd density detection, allowing for an alarm if a train platform reaches, for example, 80 percent of its capacity. Line checking, such as in airports, is another area that can greatly improve operational efficiency, Otterspeer said. “One of the things that can reduce false alarms is head detection. The shape of a human head and its relation to the shoulder is always a certain geometry. Detecting that greatly reduces false alarms from dogs or other animals.”
With the right camera placement and good software, you can do high-quality counting anywhere, Li said. “More sophisticated counting applications can be used for queue management to determine average waiting times.”
Counting and crowd management are slightly different, Lim cautioned. “Counting is expected to be more than 90-percent accurate while crowd management looks at the speed of a crowd formation and area of formation. Count accuracy in crowd management is usually not expected to be more than 85 percent.”
For Coastalwatch , people counting is most practical from a high-mounted camera in an area where background noise is less likely, said Tim Chandler, President of CoastalCOMS Division. “Our focus is to assign a risk variable to a risk index — the ‘count' or number of people is assessed to indicate that the number of people has significantly increased or decreased over a predefined amount of time. Our system ends up reporting 'load' rather than 'count,' which works best in beach areas where safety is a concern.”
Directional detection and tampering detection are basic functions that many cameras include and most customers enable, Otterspeer said.
“Directional movement detection is used for triggering alerts when a person or vehicle is moving in an area and direction that they should not be moving in. This is used, for example, in protection of critical infrastructure and in airports,” Ashani said.
Real-life scenarios for directional detection involve perimeter surveillance and wrong-way movement detection for vehicles and people, Bharadwaj added. “This is a motionbased algorithm; false alarms arise when the line drawn includes objects such as trees that move due to wind.”
Directional detection can also be applied to beaches as a further indication of risk. “The ability to draw a virtual line in the sand and see if folks are stepping across the line to move into or out of the water is useful, as it can modify a risk calculation's results,” Chandler said. “The idea is to support the risk manager with useful decision support metrics that they can evaluate in real time, especially for remote or unmanned areas of responsibility.”
Directional movement detection can be very accurate, so long as the environment is not overly crowded. Outdoor scenarios are more challenging as there are environmental conditions that can lead to false positives or negatives, Ashani said. “Modern VCA algorithms are able to robustly handle outdoor scenarios in many common cases. Another challenge is the ability to distinguish between target types. For example, the system can fail to distinguish between a group of people and a slow moving vehicle if the algorithm is not sophisticated enough.”
Typical sources of false alarms are moving shadows, changing lights and incorrect object/target classification. Each of these can be minimized by developing additional algorithmic components, Ashani said.
Like with all video analytics, tampering can be a simple application, Li said. “The system can tell if someone has sabotaged the camera or covered the lens. More sophisticated systems can also tell if the system can see clearly. So even if the camera has not been tampered with but has lost focus or can't see because it's raining too hard or if the camera has moved due to vibrations, the system will let the operator know.”
Tampering detection is a must-have for any surveillance installation. A tampered camera directly defeats the purpose of video surveillance, Bharadwaj said. “Accuracy depends on design. A good solution needs to detect tampering due to camera defocusing, blocked camera and view change. At the same time, it needs to disregard camera shakes/vibrations due to environmental conditions.”
Camera-tampering detection is applicable to any surveillance camera and enhances the operational readiness of cameras in surveillance installations, Ashani said. “For some solutions, camera tampering is not limited to only video loss or image blocking but also detects insufficient lighting or oversaturated images which result in poor video quality.” This application is generally very accurate and generates very few false alarms, Ashani added.
Object loitering and object removal are essentially the same thing. The targeted object is identified, but the alarm goes the other way around, Otterspeer said.
At the moment, object removal rules are probably most effective in places where the traffic is not too dynamic, Lim said. “Object loitering is not as magical as some would claim. If a camera is placed over a crowded airport and a crowd forms up covering one another and an unattended bag, there is no way the VCA will work.” Successful applications are likely in museums and exhibitions rather than airports and transport terminals. “I've heard stories about a public transport operator that tried to implement unattended-bag detection for trains, buses and even the stations. It was a complete failure,” Lim added.
Object removal detection is not practical if the object in question is too small, not in a well-lit area or occluded from the view of the camera for very long periods of time, Ashani added. “There was a large warehouse that installed 150 cameras, all of which carried video analytics. The customer wanted to be notified when cargo went missing, and to be able to use forensic search to find the cargo. This was not that difficult,” Otterspeer said. “However, the director of that establishment wanted to know when one little box of a cellular phone went missing, and wanted the system to follow it through all 150 cameras. That was simply not possible — and still isn't — with the current state of the technology.” Once the director was shown how to set up the right detection lines and proper rules, he was still impressed by how much it could help his operations.
Baggage abandoning is a different matter, as the system needs to understand when one object splits into two and establish a connection between the person and baggage.
Boxed solutions are generally not tailored for specific applications. If a customer is looking for one niche solution, it may still be a centralized solution because of the flexibility. It will also be more expensive because the system needs to be trained by engineers, Otterspeer said.
However, meeting customers' real-life needs is an increasingly popular requirement, Li said. “Solutions tailored for industries, ranging from oil and gas to prisons, maximizes value and reliability. For example, solutions for banking provide detection of skimming devices on ATMs, and solutions for airports provide metering of aircraft to tell precisely how long each is parked at the air bridge.”
Another example would be how people counting is used in coastalarea management. “For safety applications, people counting seems to be most interesting for remote ‘pocket beaches,' which are areas that are often off the beaten path and unguarded by lifesavers,” Chandler said. “If a large number of people suddenly appear in an unguarded or remote beach area, that may equate to higher risk if the ocean or water conditions also match up. This type of VCA, which requires accuracy within a range rather than identification of a single human form, is a great example of VCA working in tandem with business rules and VMS-actuated work flows.”
The power of analytics is greatly enhanced when alarms or events are correlated with those from other functions, such as access control or video management. A unified security platform allows the end user to view information from all the different systems, correlate it and report on it through one interface, said Jumbi Edulbehram, VP of Business Development at Next Level Security Systems. “The combination of this data enables the user to have a fully comprehensive view of security and business operations, and that is what the end user is seeking.”
The challenge in integrating with VMS is that suppliers are focused on storing information and displaying it, Li said. “They're not familiar with the sophistication of VCA metadata, so they don't have the ability to display it. For example, the VCA software may have a function that allows the user to track an individual and know which camera he has passed through over a period of time. However, most VMS systems are just focused on showing raw image scenes; they don't have the capability to ask complex questions nor to display the answers.” To complicate matters, most metadata today is still proprietary, but fortunately there will be a standardized set of metadata in ONVIF 2.0, making it easier to interact and to integrate VCA into other systems, Otterspeer added.
User interface is another problem area. “If the VCA software allows the user to go back to the beginning of an event when the need arises, the metadata to achieve this can be requested from the video analytics system,” Li said. “However, if the VMS's user interface doesn't have a button to activate this function, the user can't get this information.”
VMS integration, thus, requires software developers to work closely to ensure all functionalities can be accessed through a common user interface, and major standards bodies need to pick up their pace in addressing their clients' needs.