Surveillance VideoWall with OpenVidu (part1)

Today we talk about a project that has been outstanding from the beginning, but once in production lacked of enough consistency for an intensive use. 

If we have at our disposition a huge amount of surveillance IP cameras from a city, we know the GPS location, we can develop a solution that will display the N nearest cameras to a certain GPS point. But when talking about media delivery things can get messy.

The cameras

The cameras are deployed all around a big city, there are more than a thousand IP cameras. Many of them are property of the final client and many other are from third parties that gives explicit permission to consume the cameras media flows to the final client. As expected because of the big number of cameras and tenants the cameras are of many different types and models, and have different properties -codecs, profiles, network capacity, etc.-. Furthermore those details are completely unknown before the connection. 

So any platform that want to make use of all these cameras must be compatible with most available cameras in the market. And those incompatible should be well defined, so the platform won’t be affected if an incompatible camera is requested.

The challenges of media delivery

We find two challenges that we’ll detail: 

  • Concurrent access to media flows
  • And Codec compatibility

Concurrent access to media flows

Something that we usually forget when we work in media delivery is that in order to distribute the media from a camera to different users is that for each user the camera has to open at least one exclusive communication channel. Therefore if there are 10 users trying to obtain the media flow from a camera, this camera will have to set 10 different tunnels. This is complex and most used cameras won’t support more than a few simultaneous tunnels. However even if the camera could support those simultaneous tunnels the bottleneck is the bandwidth, because the camera will be sending media packages through all the tunnels using the same network capacities, thus the probability of packet being lost an having delay multiply by 10. And over GPRS or WI-FI networks this would really be a problem.

In all likelihood this issue may be detected once in production because it has little impact on functionality until there are many concurrent users.

Solution

The solution is quite straightforward, use a media server that is capable of multiplexing the media flows. The media server should be placed in a machine with good processing capabilities but more important with enough bandwidth. By taking this approach the camera will only send one flow to the media server and the media server would multiplex that flow and serve it to the users, in other words the media server will be the one maintaining all the communication tunnels needed.

Codec compatibility

Supporting all the different cameras in the market it is complicated and costly, and knowing that we are talking about a web platform doesn’t simplify it. Browsers -that comply with WebRTC standard- have limited codec compatibility and in many cases we will have to transcode the media to be supported by browsers. The trascodification process is “CPU expensive” so most cameras can’t run it themselves and will only provide media with the pre-set codec -in some cameras the codec can be chosen within a range -.

Solution

In this case the solution is the use of a server or proxy that could transcode so the user -browser- can “understand” the media flow it receives. As the transcoding is a CPU expensive process, the machine where the server or proxy is deployed should have enough processing capacity for the expected number of concurrent flows. 

THE SOLUTION

In this case Kurento was the first option to cover both challenges as it fits 100% by definition with the use case -multiplexing of signals and transcoding-. But some new challenges appeared:

  • It isn’t simple to scale -manage kurento clusters is quite difficult-.
  • It requires deep knowledge of Kurento components to optimize its behaviour and it also needs an specific Java development to manage the Kurento pipelines in order to add new webrtc-endpoints each time a user request some media flow.

So we proposed the use of OpenVidu PRO that wraps Kurento and simplify its use and provides out-the-box Kurento clustering. This way each camera will be represented as a OpenVidu session where users will connect as subscribers to consume its media flow.  OpenVidu PRO simplified the management of the media flows with minimal impact on the rest of the project and big improvement in the user experience.

But all that glitters is not gold, and other challenges surfaced mostly related to the displayed quality of video.