When it comes to storing large data files at high speeds 24 hours a day, storage systems must be protected from errors that can affect recorded images.
When it comes to storing large data files at high speeds 24 hours a day, storage systems must be protected from errors that can affect recorded images.
Extended storage devices are complicated, with different media and protocols. "Not all external storage devices are fully compatible with all DVRs and NVRs, despite nominally using standard network protocols," said Piotr Dolecki, Product Manager of Security Products, Siemens Building Technologies.
The solution is to test all proposed systems before using them. "There are lists of 'supported devices' available through DVR and NVR manufacturers," Dolecki said. "However, given that NAS and SAN devices are standard IT products with typically short life spans, new products supersede old products quickly, and the list of supported devices needs to be properly maintained."
Another problem is the speed of throughput for NAS and SAN devices. "This is particularly important in real-time recording applications and when using megapixel or HD cameras, which generate significantly more video data than classical resolution cameras," Dolecki said.
Video data is also constantly being written into extended storage devices. "The hardest problems to address, with respect to video surveillance storage, lie in the continuous nature of the recording and the sheer volume of data produced," said Clifford Cox, Product Manager of Digital Video Systems, UTC Fire & Security. "Video systems transmit large volumes of data continuously. Tuning storage systems for surveillance workloads is therefore very different from tuning them for data processing."
Another pain point is the network infrastructure connecting the NVR or DVR to the extended storage system. "Issues like available bandwidth and quality-of-service need to be addressed to ensure reliable data throughput," Dolecki said.
Education is key to setting up optimal storage systems. "Many of the problems associated with providing extended storage systems in video surveillance applications stem from certain assumptions, or lack of technical knowledge of the solutions available and the underlying benefits of each solution," Howie said. "The only real way of tackling this issue is through education of system integrators and end users on the capabilities of various forms of extended storage options."
System integrators must have thorough knowledge of installing storage systems. "The more proprietary the system, the more dedicated skills and training will be required. NAS and SAN systems based on iSCSI and Ethernet will be much easier to deploy and maintain," said Lee Caswell, founder and CMO of Pivot3. "Also, scale-out systems that allow adding capacity and performance to an installed system without interrupting video capture and retrieval will minimize configuration time and upgrade upheaval."
The project's density affects the extended storage system. "The level of complexity relates to the size of the system to be deployed and the requirements of the customer's environment," said Jeff Whitney, VP of Marketing, Intransa.
Some installations use the same network for both security and business, which 
carries risks. "The physical security system, especially video, can have an appreciable negative impact on the performance of the network and storage infrastructure," Cox said. "These is sues must be mode led and accounted for in the design phase of the project. Nightly backup windows and scheduled outages may not be acceptable in a physical security environment, where 24-hour operation is assumed."
Adequate planning is required by experienced system integrators to ensure the proper storage of video. Many problems may occur beyond simple hardware failure, and it can place at risk the data that is being stored. "Two of the most prominent issues that need to be addressed is stable recording and dropped frames," said Jack Yang, Senior Manager, Product Planning and Marketing, Promise Technology. When recording is unstable, it can leave gaps in the recording, which may become a big problem if the few seconds that are missed is critical to the overall clip.
Frame dropping is also a common occurrence, where random frames are not written onto the hard drive after digitalizing the information. Errors may also occur during the overwriting of data, since data is both being deleted and written at the same time. To combat these issues, an optimized firmware is used to stabilize the device and to protect against hardware failure.
Bad Sectors
A bad sector is a section of a disk that is damaged and cannot record data. Similar to a scratch on a CD, it causes the data in that one section to become irretrievable, even if the overall hard drive is unaffected. When using redundant array of independent disks (RAID), the recorded data is distributed to many different hard 
drives, so it does not pose too big an issue, especially since the parity can help rebuild lost data.
But if the bad sector occurs at the header of the data file, the whole file will become corrupt and unreadable. The lost file can only be regenerated if RAID 6 is in use. In the case of mirroring, the bad sector is copied onto the spare, which causes all subsequent copies to be faulty. "Predictive data migration (PDM) is used before bad sectors become a big threat to the file, but not all bad sectors are caught. Sometimes, they are not found until the disks are individually read for data retrieval," Yang said.
A RAID 5 device would only allow one bad sector, whereas a RAID 6 device would allow two bad sectors for each data file, since it has two parity disks. This is sometimes considered an over precaution, since the chances of finding two bad sectors is minimal.
Predictive Data Migration
PDM is a firmware feature that monitors the health of a hard drive. As soon as it senses a problem, it begins to clone all the data on the faulty hard drive onto another, healthy hard drive. "This happens before a RAID device goes into degrade mode, which means that the RAID condition is still normal, but not optimal," Yang said.
It takes on average five hours to clone a 1-terabyte hard drive and it does not affect the performance of the overall RAID device. If, however, another hard drive fails during the time it takes to rebuild the original disk, the whole RAID device could go into critical mode or crash, depending on the RAID level, and data could be lost.
Not all RAID devices support PDM and additional protective firmware, which is why it is important to pay special attention when selecting an extended storage system.
Future
The current storage technologies used for surveillance are similar to the storage technologies used for IT. As developments are made in the IT sector, the storage 
infrastructure for surveillance will improve. For example, attention has shifted from NAS devices to SAN devices. "A year ago, NAS devices were extremely popular," said Bob Mesnik, President of Kintronics. "These days, computers are getting bigger with RAID incorporated right into them."
Different compression rates increase bandwidth and capacity requirements. This affects which interface is best suited for a project. "At the moment, storage is mixed and there are many hybrid devices. This is because there are too many different environments, causing it to be chaotic right now," Yang said. "But in general, we still see 50 percent of the market using DAS systems because they still rely on analog cameras and DVRs, and 50 percent using SANs due to the IP uptake."
One problem is the amount of space available on a hard drive. "We are seeing a continuous increase in available storage size and a corresponding decrease in cost per gigabyte. Manufacturers are working to improve the reliability of their drives while increasing the reading and writing speeds, and decreasing the power consumption," Dolecki said. "The biggest weakness of conventional hard drive technology is the resistance to mechanical shock, which is critical in mobile surveillance applications."
Solid state storage provides a solution. "Solid state storage is starting to reach sufficient technological maturity for it to become a viable alternative to magnetic hard drives in the foreseeable future," Dolecki said. "However, hard drives are still more favorable at the moment due to their low cost advantage over solid state storage."
There will also be an increase in IP SAN systems. "IP SANs may represent the best balance of performance and cost when networked storage is considered for physical security, especially where surveillance recording is concerned," Cox said.
However, there are still limitations to IP NAS/SANs. "The transmission of IP video over any form of network is limited by the available bandwidth," Howie said.
The boom in megapixel and HD cameras affect the need for better extended storage systems. Storage devices will become cheaper and more advanced, as this industry realizes the need for technology to store large video files.