Video assistant referee basics: Slomo.tv on why choosing the right codec for VAR systems requires special attention

Slomo’s technology helped provide VAR for the Tokyo 2020 water polo. Slomo has worked with FINA since 2018 to provide VAR for water polo

By Igor Vitiorets, CTO, Slomo.tv.

Video assistant referee (VAR) technology or video assisted review (the name introduced by FINA and, in our opinion, an excellent description of the video-refereeing process in any sport,) is rapidly being implemented in all sports, including those with limited budgets. Therefore, the development of affordable VAR systems is an important modern trend.

VAR is a replay system with specialised functionality. The software defined server architecture becomes virtually the only option for VAR systems. In such servers almost all operations are performed using the CPU.

One of the key operations is the encoding and decoding of uncompressed incoming video. This process is performed by a part of the software, called the video codec. The video codec is defined by two parameters: the compression standard and the way compression is realised.

Points and principles

Let’s talk about the main points and principles that a codec for VAR systems should be selected. In order to choose the “right codec” it is necessary to establish the evaluation criteria that allow us to make at least an initial selection among the codecs.

The first criteria that comes to mind, and definitely an important one, is the quality of the recorded video produced by the codec’s “encode-decode” cycle. Unfortunately, we know several cases when referees working with VAR systems could not distinguish the details on the general camera plan. This is unacceptable for accurate refereeing.

Since the VAR system is often a part of the broadcasting complex, the video from the referee’s monitor has to be outputted to the broadcasters. In most cases, VAR systems receive signals from professional TV cameras, so the outputted video must be of the same high quality.

The second criteria is the bitrate, which defines the amount of disk space required for one compressed frame. Developers of VAR systems strive to reduce this number to save storage space.

It would seem that these two parameters should be enough to choose a codec. However, we should not forget that the codec is used in a software-defined server, which means that it must be “light” on the CPU. The codec should consume as little operation time of the main processor as possible, because the server works with many input channels, recording all of them, and playing back up to six channels simultaneously. If the codec allows us to compress the video without distortion, provides low bitrate, but, as a consequence, consumes a lot of CPU resources, it is not suitable.

Pay special attention

When choosing a codec, pay special attention to the fact that a codec that works perfectly with a couple of video channels and provides good image quality will not necessarily show the same results when working with all channels because of too-high CPU load.

As a part of the multichannel recording and playback systems, the codec must provide the ability to instantly start a video playback from any timecode (TC). This function is one of the main tools of the VAR referee. It should not add additional load leading to exhaustion of the CPU’s performance resource. Otherwise, the picture will lag, freeze, and in the worst case, some input frames will be lost.

We have outlined four main parameters for selecting a codec, based on which some conclusions can be made.

Currently, low bitrate is provided by interframe codecs, such as H.264, H.265, MPEG-2, and MPEG-4. However, these codecs have a high CPU overhead. They also have problems with immediate frame positioning. These codecs are not optimal for VAR systems.

It is worth emphasising that due to their peculiarities and standardisation, such as bitrate limitation, interframe codecs sometimes cannot provide acceptable image quality when there are a lot of small moving details. Theoretically, it is possible to create a codec that can provide sufficient quality in such situations, but it will require even higher CPU load.

For this reason, the basic codecs for video refereeing should be considered the intraframe codecs. Nowadays the choice of such codecs is quite large, many of them are used in TV production: MJPEG, Apple PrоRes, Avid DNxHD, AVC-Intra, TICO, and JPEG-XS.

Due to the fact that these codecs are used in broadcasting, they have configuration options that provide acceptable quality. However, codecs with different settings have different implementation and usage features. Some require obligatory licensing from the manufacturer, others require high CPU capacity or have implementations that do not provide the desired quality. For example, an excellent AVC-Intra codec at a very low bitrate of 100Mbit can provide excellent as well as poor quality, depending on the implementation.

From our experience in video refereeing we can say that the most suitable codecs for VAR tasks are high quality Apple ProRes (SQ or HQ) or DCT-based codecs with high bitrate.