Thought Leadership: Mobile Viewpoint CEO Bais on the rise of the AI robots

Artificial Intelligence (AI) is becoming more relevant throughout the production chain for live sports. Here we talk to Michel Bais, chief executive at Mobile Viewpoint, to get his views on just how much of that workflow and which sports could one day be taken over by AI algorithms.

How can AI automate the whole video production chain?
When you watch a sports channel 95 per cent of all the sports are captured in a certain way and according to a well-evaluated workflow, which could be automated.

Within Mobile Viewpoint we worked on the creation of video production automation software over the last two years. An important part of automation is the detection of players, balls, field lines and goals. Analysing many techniques for image recognition indicated that AI algorithms for image and object detection could solve those detection challenges.

Many [examples of] automated or lane-assisted car software also uses AI to steer a car and even wait for traffic lights. AI is very good at automating repetitive tasks like finding white stripes, people’s faces and, off course, number plates.

Mobile Viewpoint worked on fine-tuning AI algorithms used in other fields of operation to detect faces of presenters, sports players and balls to street PTZ cameras to the right location. In combination with AI algorithms creating a virtual director, it is possible to produce a soccer or tennis match without the need for a real director or camera operator.

Next to camera and director automation, those AI image recognition tools make it possible to dynamically replace advertisements on boarding and create summaries based on the AI detection of highlights. AI already plays an important role in post-production, where it is used to find specific people or quotes using speech to text.

However, due to the nature of AI automating repetitive tasks, it would be difficult [to use it] to automatically create social videos during half time, or, for example, 100% error-free subtitling.

What is AI’s value in live sports?
Due to the growing availability of low-cost hardware doing all the hard AI work, it is possible to create low-cost portable production kits that will enable the production of sports that [do not] make it onto TV that often, like volleyball, cricket, and lower league sports.

At the moment only 10% of professionally played sports make it onto TV. AI unleashes the other 90% and makes it possible to create sponsored free to air TV and internet channels.

AI also makes it possible to put up automatic cameras in more remote locations like you see with rally racing and cross country skiing events. The set up of AI camera’s tracking people and cars makes it possible to receive feeds in the OB van already annotated, at a lower cost. Those cameras will perhaps not make the best shots yet, but [they can make] very interesting [shots] and [the quality is] good enough for OTT channels and other internet-based content distribution platforms.

What other uses does AI have in sports broadcasting?
As said, AI is good for repetitive tasks like you seen in an average sports game shown on TV. This also counts for the studio production of a match. During half time we see the commentators discussing the match, captured by several camera’s; [this is] easy to automate.

With the help of AI, it is easy to track people in a studio and control PTZ cameras to get the right shots. Based on gesture recognition and who is talking it is possible to get the right shot of the right person, without the need of a director.

What does AI mean for leagues and clubs? Who is or will be using this and why?
For many leagues, AI creates the possibility to start broadcasting on an internet channel Like Facebook. It is also possible to give room and a podium for the lower leagues within the federations. Examples are female soccer and lower league rugby. Enabling automatic summaries will also result in attractive content from youth teams, for instance, [that are] performing well.

Where is AI for live sports right now, and where will it be in five years time? What needs to evolve or change (including people’s attitudes) to enable that to happen?
Video AI for sports is maturing quite quickly, but the audio and human-interest parts are still developing. Watching a game is more interesting [when you have] a commentator, and [I don’t mean] the one you get with FIFA on the PlayStation; you want to hear the real stuff. This is not yet automated but will happen using remote commentators or even text to speech engines.

Looking at the video part of AI sports production, people at home will rapidly get used to it and perhaps recognise the automation, but [won’t mind because it will bring them] content they were not able to see before.

In five years’ time, you will see more sports completely [produced with the use of] AI tools. However, the big games will stay humanly directed and produced since it will still take quite a while for an AI algorithm to take over all the brains waves of a real director.