Changing the game: How AI avatars will take sports broadcasting to the next level

By Grigory Mindlin, general manager for broadcast, disguise.

Whether you’re building an immersive fan experience or want better game analysis, I believe there’s no better playground for extended reality (XR) than the world of sports. And I’m not the only one who thinks so. In the past few years alone, Sony purchased Beyond Sports to create immersive content; the NFL introduced mixed reality elements into its broadcasts; and the Premier League announced it would use AR to improve viewer engagement via an app built using the same technologies as Pokemon:Go.

At the same time, we’ve also seen AI start to completely change the way we work. NFL used Respeecher, a deep learning audio tool, to recreate the voice of deceased football coach Vince Lombardi for the Super Bowl; and Disney+’s latest Marvel series Secret Invasion opens with an AI-generated title sequence. That’s why it’s important that we look at how AI and XR can work together. This is especially true for AI-driven characters, which will take sports broadcasting’s application of XR into another league entirely. From next-gen data to deep fakes, below are the main ways I think they will change the game for the future.

Deep fakes and metahumans

Imagine if anyone could drive fully virtual characters with their movements using AI. For sports broadcasters, this would allow your studio audience to step into a MotoGP car and try a circuit out for themselves. Or let real-world presenters seamlessly interact with digital football stars. Or even let fans at home find a digital seat to a game using their avatar – giving them a front-row view that would have otherwise cost thousands.

At disguise, we’ve been fascinated by these types of possibilities for years, which is why we’ve spent a lot of time exploring AI-driven characters. We recently even collaborated with a company called Move.ai to launch Invisible – a tool for markerless motion capture that is perfect for creating avatars, as well as digital characters that can be captured in-camera on a virtual set.

We’ve also experimented with tools like Adobe Firefly. With Firefly, you can create generative content for your production or replace existing content. That means you can take an animated or physical character’s performance and completely replace the appearance with a different character altogether. This opens up the possibility of metahuman technology bringing new audiences to sports broadcasts. A nine-year-old might be bored by a traditional Premier League game, but if you could broadcast another version that resembles something like Spongebob Squarepants you could bring in an entirely new audience.

While these technologies promise to boost engagement and make sports more accessible to all kinds of fans, they could power deep fakes for all sorts of purposes. But while concerns around AI are warranted, we hope to see some kind of regulation to stop it being used for misinformation. With the right measures in place what we have is a tool that can increase productivity, save money and hopefully reduce stress when incorporating XR into sports broadcasts. Will AI replace humans in certain roles? Only those that require endless repetition and little human contact. Anything based on creativity, expression and communication will thrive in a world where those things can be enhanced by AI.

Enhancing the story behind every goal

Of course, even the best metahuman wouldn’t have much impact if there wasn’t a great story behind it. That’s where data comes in. Data is integral to sports broadcasting, how it’s presented helps tell a story to the audience and create drama.

For many sports broadcasters, the best way to present this data is by using XR graphics like scorecards. But XR graphics are also often limited by a simple fact: the majority of data in sports is created by human movement that happens in real-time. That makes it impossible to generate very impressive custom graphics as they’d need to be built from scratch.

If this could be supplemented by AI, that would change. We could generate much more engaging visuals out of the data for an enhanced audience experience. Competitions like the NFL and the Premier League, for instance, could benefit from AI gathering data or footage from past events, helping to tell the ongoing story to a loyal audience. AI could even be used to track the movement of each player on the field or vehicle on the track and provide viewers with unparalleled insight into their performance, through all kinds of rich statistics from ball trajectories to kick distances.

Of course, AI can’t always be relied upon to get it right, at least not yet. But within the next few years, I believe data specialists will help sports broadcasters wield AI as a tool to gather data throughout a sports game. Broadcasters will then use that data to boost audience engagement with graphics that can tell a better story for all.