Storage is one of the biggest costs of a surveillance project, making it a precious commodity. Network bandwidth is also scarce, making compression essential to surveillance monitoring. Unfortunately, compression can also undo clear images. “You could have a great camera capture images at high resolution, but the images are encoded, compressed and the final product is less than what was originally captured,” said Benjamin Solhjem, PM of Motion DSP. “The goal of our software is to reconstruct. That’s not always possible because the image is so degraded, but it can reveal details that were not evident at the beginning.”
Crunch Time: Compression
Storage is one of the biggest costs of a surveillance project, making it a precious commodity. Network bandwidth is also scarce, making compression essential to surveillance monitoring. Unfortunately, compression can also undo clear images. “You could have a great camera capture images at high resolution, but the images are encoded, compressed and the final product is less than what was originally captured,” said Benjamin Solhjem, PM of Motion DSP. “The goal of our software is to reconstruct. That's not always possible because the image is so degraded, but it can reveal details that were not evident at the beginning.”
Compression renders megapixel's added resolution moot. “For enhancement, we've got cameras deployed that are 1.3-megapixel for 1080P or 720P, but you can't easily transmit that resolution ‘native' over a cellular network, which many cameras are on for backhaul,” said Dave Gorshkov, CEO of Digital Grape and Chair of the CCTV and VCA Technical Standards Working Group for the American Public Transportation Association. “You can use H.264 to squeeze the video signal hard, then interrogate the signal, but it throws away the resolution of images necessary for high-speed VCA capabilities.”
Banks and stores that are robbed may already have video systems installed, but find that poor resolution defeats forensic purposes. While it's possible to cram a month's worth of video on a single hard disk drive, the resulting images are not usable. “At that point, it's almost too late,” Solhjem said. “When it's crunch time and you actually need the video, it's not good.”
H.264 is one of the most common compression formats, which was designed primarily for entertainment— set-top boxes and mobile phones — rather than surveillance. This means users should pay attention to how many I-frames are used and latency in I-frame timings, which affects bandwidth and storage. “A 1-megapixel image has at least twice the resolution of analog,” Gorshkov said. “However, once you send that over the network, it's going to be squashed a lot and have latency introduced into it due to the compression and decompression functions.”
Storing images at the best possible quality will yield better data for image enhancement algorithms. “When we were developing the video surveillance and VCA standard in the U.S., local storage of high-resolution images was required,” Gorshkov said. “It's not the compressed image operators observe, but the stored high-resolution image that is used for legal proceedings. Most network cameras now have dual or even triple streaming, which is wonderful way of doing that. Local hard drives are also not expensive.”
As storing full-size images is out of the question for most applications, selecting a lossless compression and storage at the highest resolution yields the best evidence. “The effort to save space on a hard disk drive uses a compression scheme that's great for space, but it's not great if you need to pull out key information from the video,” Solhjem said.
Legal Implications
Image sharpening tools can certainly deliver evidence-grade images, albeit with limitations from resolution and compression. However, not all legal systems accept digitally processed images, so the enhanced images are used as tools during investigations.
Several legal trials in the U.S. and U.K allowed for enhanced images. “One time, the Metropolitan Police in London used our software to submit evidence for trials,” Solhjem said. “To my knowledge, it's never been thrown out.”
Some government bodies verify whether technology is suitable for police and government use. The UK Home Office uses the Image Library for Intelligent Detection Systems (i-LIDS) test to determine whether VCA is effective in various environmental and operational conditions, Gorshkov said. These tests help reveal possible shortcomings with the algorithms.
In the U.S. and Canada, Frye and Daubert hearings assess whether a new technology passes muster within the scientific and forensic community. “As long as they're not putting things there that do not exist, it's admissible in court,” Solhjem said.
Best Practices
Image sharpening software is great, but its performance is aided by good system design. This considers compression, storage and user demands.
Raw uncompressed video or a lossless compression gives image enhancement software more data to analyze. “Do a test, record some footage and see what it looks like,” Solhjem said. “You definitely don't want to plug-and-play and forget about your video system.”
Customers should also think critically about what they want their video system to do. Checking if someone came into a reception area is much easier than identifying individuals at border control. “Depending on what you do with the system, it comes back to Design 101,” Gorshkov said. “It's understanding what it is you want from your video surveillance and VCA system, as the systems' reliability and performance requirements are very different.”
Camera placement should be optimized for the monitoring purpose — that means keeping the camera away from direct lighting that washes out image detail. Image quality also needs to be a priority, which should be as good as the customer can afford. “If you really need video surveillance, is it worth the money you're investing?” Solhjem said. “If you're willing to invest this much, are you willing to invest more, in case something happens?”
At the end of the day, image sharpening tools can help restore pictures to their original glory. However, much image degradation can be prevented by better compression techniques. It is not cheap to store raw video, but lossless compression with multiple streams is a step in the right direction. While Hollywood presents pure fiction, an ounce of planning plus image sharpening keeps surveillance footage from going to waste.