People enter the main square, a computer screen at the city’s operations center beeps an alert. Analytics engines combining inputs from video and social media predict that a riot is about to erupt. Looking for people potentially carrying weapons, the cameras at the scene automatically detects people carrying backpacks from within the crowd. This is not science fiction: it is part of the capabilities recently demonstrated in Singapore as part of its safe city pilot project. Once an emerging technology, video content analytics (VCA) is becoming more mainstream.
People enter the main square, a computer screen at the city's operations center beeps an alert. Analytics engines combining inputs from video and social media predict that a riot is about to erupt. Looking for people potentially carrying weapons, the cameras at the scene automatically detects people carrying backpacks from within the crowd. This is not science fiction: it is part of the capabilities recently demonstrated in Singapore as part of its safe city pilot project. Once an emerging technology, video content analytics (VCA) is becoming more mainstream.
The use of video cameras and video content analytics (VCA) for city surveillance is no longer a novelty. As safe/smart city projects become more prevalent, the traditional use of video surveillance and VCA is changing. Previously restricted to security or safety alone, the future of VCA in a safe/smart city lies in connectivity with other systems, enabling a more dynamic and comprehensive, real-time situation picture, thus allowing operators to better handle events and take full advantage of video footage.
Zvika Ashani, CTO of Agent Vi, elaborated on the use of VCA in a city environment. He pointed out that fundamentally, video analytics are used for three main objectives: security, safety, and city management. Security analytics are based on a predefined organizational security policy and include capabilities such as detecting vehicles parked in no parking zones, intrusion detection, unattended baggage detection, and loitering. Safety analytics can identify crowd gatherings, blocked emergency exits, or a vehicle stuck on a railway. Management analytics uses the data from video cameras to collect statistics used for city planning, such as traffic flow, vehicle counts, speeds, etc.
New innovations coming from Singapore
In 2013, Singapore initiated the Safe City Test Bed project. Its R&D initiative is to develop a safety and security industry with innovative capabilities in the city state. Karianto Leman, Head of the Situational Awareness Analytics Program in Singapore's Institute for Infocomm Research (I2R) — a member of the Agency for Science, Technology and Research (A*STAR) — shared some of the key contributions of VCA in smart city security. For example, I2R has developed a system that detects aggressive human actions. The system automatically extracts motion related features and feeds them to an artificial intelligence engine. The engine has been trained with video samples involving aggressive human actions to detect an occurrence of such an incident in real time. In applications, the system can be configured to detect gang fights, rioting, rampaging, etc. Another possible use is to protect security guards from being ambushed. In the safety and city management domains, I2R developed crowd analytics that could work robustly under very crowded conditions. The system can extract from real-time information such as crowd size, walking directions and speeds, and the number of visitors over time. It could also mine these metadata to unveil insights such as popular paths, point-of-interests in retail or exhibition spaces, crowd behaviors, and other statistics.
Key technologies to this are accurate human detections and tracking even when crowd density is very high. This overcomes the pitfall of most video analytics systems where performance drops drastically in crowded conditions. Applications of this technology include crowd sensing and activity profiling at public places such as train stations and shopping malls, allowing, for example, estimations for waiting time in queues for taxis, banking services, etc.
.jpg)
It's all about connectivity
One of the principles behind safe/ smart cities is creating an interconnected environment. In this setting, different systems are combined to ensure the city's safety and efficiency. The future of VCA in safe cities is in connectivity with other systems and combining information from different sources to create a more detailed situational awareness picture. “Security applications are still very much in focus. We are seeing several new VCA applications areas in city safety, efficient use of energy and other public resources, street parking management, and many others,” said Dr. Sadiye Guler, founder and President of intuVision. "For example smart street lights that are automatically brightened when there are only few people on a street to increase safety of the citizens also helps save electricity. The emphasis is on smart city solutions that combine VCA with other sensors and systems to develop city wide combined solutions.”
In addition, “Safe cities are more concerned with making traffic and people flow more efficiently together with improving safety,” said Bill Flind, CEO of Ipsotek. Therefore, he added, “VCA shouldn't be visualized as working alone but as part of the wider situational awareness system providing managers with suggested responses and action plans”.
One example is the image recognition and incident detection system, developed by Ipsotek, for Transport for London (TfL), the department in charge of keeping London's transportation systems flowing. When a road in central London was blocked by a vehicle that had lost control across the carriageway, the system identified within 30 seconds that this was an incident requiring rapid intervention. Predefined contingency plans alerted the relevant TfL staff allowing them to take immediate remedial action including re-sequencing of traffic lights to divert traffic away from the incident spot. The ability to control rapidly developing build-ups of traffic around accidents, keeping traffic flowing through the use of an “always-on” system significantly reduced journey times in London.
Tool for post and pre-event management
According to Jamie Wilson, Security Marketing Manager for EMEA at NICE Systems, the power of VCA is not just in post-event investigation and scenario reconstruction, but “also in the ability to know sooner that an incident could potentially take place, or is starting, improving both the speed and quality of response.”
A good example of this would be VCA reporting crowding and movement through a public space combined with information about an event or sentiment analysis from a social media tool like Twitter or Facebook. This multi-source situational awareness would increase efficiency and help facilitate effective and timely decision making and faster deployment of necessary resources — a police car, a fire engine, or even a garbage truck.
This allows for more efficient management of security forces. The limited resources of security personnel could be optimized by assigning them to the places that require them more urgently. By sharing video footage or images containing the incident to the security officer to validate the alarm, they can get an understanding of the situation.
“Scenario-based video analytic systems accurately recognize complex situations by evaluating and combining multiple conditions. It is this combined analysis that delivers the detailed understanding of the operating environment, and it is this understanding that dramatically reduces false alarms even in complex environments, leading to VCA becoming a trusted information provider to city surveillance projects,” added Flind.
.jpg)
Technological advancement In VCA for city surveillance
City environments are complex environments for VCA to tackle. The challenges include analyzing crowded areas, changing lighting, weather conditions, distance of objects from the camera, the need to seamlessly move from camera to camera when tracking an object, etc. These are but some of the obstacles facing VCA vendors. Despite the challenges, the growing technical capabilities of processing algorithms and advancements in the field of data storage will facilitate a more widespread use of video analytics.
As the number of semantics detected by video analytics grows, so will the efficiency and power of forensic searches. Security officers will be able to conduct faster and more detailed searches. The system developed by I2R, which can detect high-level human semantics could pick out from a human crowd a person carrying a backpack or sling bag. This can be performed on general video surveillance camera image resolution. In addition, the system can detect people wearing sunglasses, headdresses, clothing patterns, etc. It could also estimate the height of the person that the system is tracking. The continuous improvements in data storage will facilitate growing amounts of video footage being stored and analyzed. This means cities will be less restricted when they need to prioritize which information to keep and which to delete.
Too much information to digest?
“The mass of data has become a benefit as opposed to a challenge,” said Wilson. “By integrating all of the data-feeds, the operator is given a single clear picture of what is going on and can focus on using his skills to manage an incident rather than trying to interpret the data. In turn this speeds up resolution of incidents, saving time, money, and resources.” There is still quite a long way to go in intelligent video solutions for safe/smart cities; future goals include features such as automatic tracking and switching between cameras and of course reaching reliable face recognition from security cameras. With the speed at which technology is evolving, and with growing numbers of safe city projects being implemented, these capabilities will for sure be on the agenda of many vendors.