ITV Sport puts artificial intelligence in the Timeline for sports production

ITV Sport has moved its archive to a new cloud-hosted system that uses artificial intelligence (AI) to help add metadata and output multiple versions for highlights across social media, on-air use and archiving.

The broadcaster has a huge archive and wanted to leverage that media and needed a MAM that would allow them to find clips easily and then quickly make new edits. Timeline Television was brought in to look after the project to deliver an enterprise-hosted MAM system on a private cloud.

Dan McDonnell’s Timeline team has moved some 200,000 ITV Sport assets to the cloud

“We run a hosted solution at our premises in London, and we collected all of their data and moved all of their files and videos, about 200,000 assets, into a Tedial MAM system,” explained Timeline MD Dan McDonnell.

The system Timeline installed also served all the content back to users using Aspera to provide optimised, high-speed file transfers, including clip selection and basic editing. It also has several large LTO tape robots for long-term archive, plus online storage for fast access to content.

“Tedial is very, very clever in the way it decides what is going to be kept online,” so at the World Cup the system knew to keep clips from an England match online, while placing other matches on tape (which might take a few minutes to access but weren’t in such high demand). They system has been in use since early this year, and now has more than 100 users at ITV Sport across multiple sites.

It is fully integrated into the end-to-end live production system, allowing users to quickly find a clip and then send it on to an EVS suite for on-air use or to a Premiere or Avid suite for editing, or to social media from a single interface.

The archive is also easily searchable, with extensive metadata, using AI to help to generate this data.

The case for AI

According to Jérôme Wauthoz, vice president of products at Tedial, a lot of its customers in sports production are having to cope with the rising cost of rights, often driven by competition from new entrants to the market such as Amazon and Facebook, “which means that these guys are really under pressure.”

It doesn’t help that consumers are also now demanding easier, quicker access to more personalised content, often beyond traditional linear channels, so “our customers must produce more and better on the same budget,” he said.

Meeting these demands requires systems that are faster, simpler, and less expensive to use, and can

Tedial’s Jérôme Wauthoz believes AI will be broadcaster’s best buddy

automate as much of the process as possible. What Tedial offers is a system that automates metadata and content creation so that users can “just focus on creativity so that they can create more content with a higher level of quality.”

This is the rationale behind Tedial’s new SmartLive system, which was introduced at NAB and has been designed specifically for sports production as a metadata engine that can take in any data feeds and any manual logging information, link them to the video files, and then act on them to identify all the goals, penalties, red cards, or whatever you require.

This can then be automatically assembled into clips and put out to social networks. SmartLive can also automatically create highlights of the best action from the game to any given length. “We want to improve the way people are working, to shorten the time they spend to search for content using these metadata,” said Wauthoz. The AI system means it can constantly improve on what it delivers as it learns what users want.

It includes speech recognition, to automatically generate text based on the commentary track, from the live feed or existing files, using it to build up the logs for each event, “which gives you more possibilities to find the content you want.”

AI can also be used to analyse the video feed in the cloud. It can classify the action, such as a “tackle, right foot”, and uses face recognition to identify the players (also using the numbers and any logos or lettering on the jerseys to help distinguish between teams), and doesn’t require any external data feeds or media logging. “The system can be trained per sport, per league, in order that you can get all this metadata automatically in the system. It augments all the data that would have been made available before,” said Wauthoz.

“This is particularly interesting for sports where you don’t have any data feed or where you don’t have all those data logging people. You can use it and automatically generate the data into the system.”

It then uses auto-clipping of goals, red cards, or other significant action, to create clips that can be automatically published to social media.

For highlights “we have a tool that is dedicated to package storytelling in just a few clicks,” he said. It uses its information on all the actions in the match (such as corners, goals, etc) and the players, and it will create full match highlights, or for a particular player or team. Users could ask it to create a one-minute highlights package for social media, plus a 2.5-minute version for on-air playout, and then a 25-minute package for archive. All this could be achieved in seconds with five clicks. The system generates an EDL for each instance so that someone can review and approve each package, and give it a polished edit if necessary.

Wauthoz sees the key benefits of using AI, besides the automatic metadata creation, as: the ability to easily produce personalised content that can lead to new revenue growth; improved storytelling (because you can tell many more stories than before); and more efficient production workflows, allowing broadcasters to produce more and better on the same budget.

However, he was keen to emphasise that “artificial intelligence does not replace humans. In our domain, in the broadcast space, AI will be your best buddy. It will be a virtual colleague who will help you produce more and faster.”

Besides the use of AI to enhance workflows, “security was really, really important, so we had to be able to deliver secure media,” said McDonnell of ITV Sport’s move. All of the media had been located in ITV premises, and was now moving to the cloud and needed to be easily accessible.

“All of the MAM users, whether within the ITV premises or at the World Cup in Moscow, were just using a standard laptop and browser and were logged into the archive over a secure VPN” – Dan McDonnell

The result was in use in time for the World Cup, and in Moscow ITV staff “could sit on their laptop and browse all of their archives and look for previous World Cups, previous matches, find the clips and put in on to social media or into their edits or to air as quickly as possible,” he said.

Also, as it uses EVS servers for a lot of the ingest and slomo workflows “it was key that the solution we put in for ITV Sport could integrate seamlessly to their EVS system,” added McDonnell.

Another key deliverable for ITV was that the actual interface to use the MAM was an HTML5 web application, “so all of the MAM users, whether they were within the ITV premises or at the World Cup in Moscow were just using a standard laptop with a standard browser and were logged into the archive over a secure VPN.”