Do more with less

Fisheye cameras have been around for years, but real-life adoption seems to be limited, due to a limited number of offerings and VMS integrations. Things are changing for the better, as more camera manufacturers and management software providers pay more attention to the optics and user benefits. a&s looks into the latest developments.

Panoramic and hemispheric lenses have been used by photographers for decades. Nowadays, one can literally shoot images with panoramic or fisheye effects with smartphones. Still, what is common in the consumer electronics industry may be exotic in security.

The panoramic camera offers both the 180-degree and 360-degree panoramic view using a single fisheye lens or several normal lenses. “For a panoramic camera using several lenses, the panoramic image is produced by stitching the images shot by each of the lenses,” said Osborne Fang, Marketing Manager at Brickcom. “On the other hand, the fisheye camera provides the panoramic image with a single lens. Due to the extremely wide viewing angle of the fisheye lens, the image is distorted and dewarping technology is needed to turn the image into a normal one, readable to the human eye. Hence, dewarping is one of the keys to the final image quality of the fisheye camera.”

Petra Bennermark, Product Manager at Axis Communications, added, “A panoramic camera refers to either a camera that is mounted on a wall and gives a 180-degree view from the wall or a ceiling-mounted camera that gives a 360-degree view of the area below the ceiling. A camera with a 180-degree fisheye lens constitutes one technical solution to such a camera. So I would say that a fisheye camera is a subset of panoramic cameras.”

One way to envision a 180-degree view is to imagine standing in the center of an alley with your back against a wall. “Cameras mounted where your head is would be able to see the whole alley including both ends and the wall that your head is against,” said Ellen Cargill, Director of Product Development for Scallop Imaging (a division of Tenebraex).

Pluses & Trade-Offs
It is important that user s understand what they get and what they do not get from fisheye cameras. Fixed cameras, however mounted, cover narrow fields of view and multiple cameras are required to give adequate coverage of wide areas. PTZ cameras, on the other hand, require manual control and are therefore most often pointing to the wrong place at the wrong time. “A fisheye camera mounted on a wall or on the ceiling in the center of a room or area can cover a vast area with no blind spots, no moving parts, and no manual control required,” said David Myers, CTO at AMG Systems. “While not totally replacing fixed and PTZ cameras, fisheye cameras provide total coverage and reduce both the number of cameras required and the infrastructure required. The ability to follow a subject on a single camera is also of great benefit in forensic analysis”

Some trade-offs still exist for fisheye cameras. As a fisheye camera uses a specialized lens that distorts the image of a scene to fit it into a single view, circular fisheye images do not use all the pixels on an image sensor. Also, image magnification diminishes as the subject moves toward the edges of the scene despite that megapixel image sensors and digital image processing have improved fisheye performance on the edges of the field of view.

Distortion & Pixel Check
Fisheye lenses project a circular footprint and have constant or linear distortion from the center to the edge. Objects closer to the camera appear bigger, while they reduce rapidly in size as the distance from the camera increases. In other words, users could have an unavoidable drop-off in image quality at the edges. These issues might be “magnified” after dewarping. It is similar to the difference between a globe and a map of the world when attempting to provide a two-dimensional representation of the three-dimensional world. Both poles are squished, like oversized representations of Antarctica and Greenland on some maps. The same goes for a fisheye lens: the farther an object is from the center of an image, the more distorted it becomes.

Image distortions and pixel drops of fisheye cameras are hard to read from technical specs. The best way is to actually test the cameras and check the details. “When comparing, make sure that you compare ‘apples to apples' — that is, the same part of the scene, the same field of view, same distance and direction angle from the camera,” Bennermark said.

There is a lot of confusion when it comes to representing the true resolution of fisheye cameras as they, compared to fixed cameras, have a smaller optical image sensor. Greg Alcorn, Director of Global Sales at Oncam Grandeye shared some simple mathematics that can help end users and integrators check exactly what resolution they are getting. “Most manufactures use a 5-megapixel sensor, or 2592 by 1944 pixels, and the image is then cropped to the size of the lens, or a circle within a rectangle. The circle, therefore, has a diameter of 1944 pixels and the following equation is used to calculate the true resolution. The actual resolution equals the area of the circle. [In this case, the actual resolution is 972x972, which is 2.9 megapixels.]"

Dewarping
Human eyes are not used to optical distortion and dewarping is created for this purpose. Dewarping can be done on fisheye cameras or on the client application. “The achievable optical resolution of the lens and the achievable resolution of the image sensor play an important role on the image quality of the dewarped image,” Fang said. “For example, a high-optical resolution fisheye lens with the 5-megapixel image sensor can provide more detail than the same lens with the 1-megapixel image sensor. Meanwhile, the computation power of the processor determines the time it takes for the dewarping function to process the raw optical image.”

In-camera dewarping offers direct dewarped views based on user's need. User can choose the number of dewarped views of the scene and the camera will do the rest. “Dewarping on the camera lets the customer create virtual views or narrow-field windows that create blind spots similar to conventional cameras. The windows may be moved using PTZ commands over the network, and will then have latency similar to what is currently associated with mechanical PTZ cameras,” Alcorn said. “[However], dewarping on the camera does not allow the ability to retrospectively view the 360 image, which is one of the biggest advantages of using 360-degree cameras.”

Dewarping on the client side, on the other hand, allows for retrospective viewing, which enables the user to go back in time to view the total scene in its original form and then pan, tilt, and/or zoom within the 360-degree image as desired. “An additional advantage to dewarp on the client side is the ability for multiple clients to view the same image with different perspectives concurrently. There is no time slicing required as with traditional PTZ cameras or 360 cameras with dewarping onboard the camera,” Alcorn added.

However, Alf Chang, Senior Consultant at a&s held a different opinion. ”In-camera dewarping may also allow multiple clients to view the same image with different perspectives. This relates more to NVRs than where the dewarping occurs. Indeed, the mode or numbers of dewarped views can only be set by one user at the front end. These dewarped views and footage are stored at the NVR. Via NVR, it is still feasible for multiple clients to view the same image with different perspectives. ”

Dewarping in the camera or on the client side have their pros and cons. When cameras are capable of dewarping themselves, installers or integrators have fewer concerns regarding the loading on the client side because the camera is doing the processing. Dewarping on the client side, on the other hand, offers comparatively more flexible and retrospective viewing. There are many dewarping methods. “Some well-known dewarping methods include stereographic projection, equidistant projection, and cylindrical projection. Each method has its own advantages, and the implementation details of the dewarping methods have to adapt to the optical characteristics of the fisheye lens used,” Fang said. “The key is the popularity and compatibility with the VMS.”

VMS Interoperability
Most fisheye manufacturers require a separate SDK for dewarping and e-PTZ controls. This makes integration to V MS systems challenging. In order to enhance the algorithm to the fullest of fisheye cameras, a VMS with full support is crucial. Limited VMS integration will block immersive fisheye cameras from being used effectively. VMS which support multiple manufacturers or even an established standard are key for broader adoption of fisheye cameras.

Oncam Grandeye is optimistic about the potential of fisheye cameras and increased integration with VMS platforms. “Technology moves forward and image quality gets better. VMS, like everything else, is advancing. We have developed an SDK for multiple platforms that can be integrated into any clientside application. It comes with our patented dewarping technology using OpenGL and 3D modeling techniques to project a texture map of the image onto p-surface (a 3D object, generated by a computer graphic system). This takes advantage of current state-of-theart GPUs (graphic processor units), which reduces the load on the CPU. Performing dewarping on the CPU could use as much as 20 percent of the CPU's performance, based on the size of the image and frame rate,” Alcorn said.

Myers agreed. “We provide the software to our VMS partners which allows fast dewarping of the high resolution fisheye live and recorded through their viewing software.”

Brickcom is also eager to increase its technology partners. “Our fisheye camera is compatible now with more than 40 VMS solutions thanks to the open ImmerVision 360-degree viewing standard,” Fang added.

If cameras are capable of dewarping, VMS interoperability is not a concern. “As Axis does dewarping in the camera, VMS is subscribed to the already dewarped images. These dewarped images can be dynamically controlled to make it a digital PTZ camera. There is no need for VMS applications to deal with dewarping,” according to Bennermark.

Future Development
Some manufacturers are launching day-night fisheye cameras for 24-hour coverage. Also, they can be deployed outdoors. The goal is to allow fisheye cameras not to be limited to indoor installations but to wider outdoor areas.

For the past few years, the greatest advances in IP-based video surveillance have been in compression and megapixel technology. These technologies bring panoramic and hemispheric cameras up to a new level, and they can be used in more applications than before, said Steve Ma, Executive VP of VIVOTEK. “With today's sensor and lens technologies, manufacturers are able to provide high quality 360-degree cameras. We will bundle fisheye cameras and speed dome cameras as a solution for great perspectives and details, which allows users to simultaneously monitor both an overview and a detailed regional view of any given scene.”