IBC Accelerators: Changing the Game: Predictive Generative AI to take personalisation and engagement to the next level

In an IBC Accelerators first, the 2024 programme features a series of challenges under the umbrella theme of AI Media Production Lab, which sets out to explore a series of specific AI concepts in various production scenarios. This encompasses researching and developing learnings around the various AI tools and techniques in terms of how to improve creativity, bias and efficiency in this fast-moving, disruptive area of media and technology.

One aspect of this is Changing the Game: Predictive Generative AI, proposed by Champion Verizon. This case study will look at AI as a production tool for live sports and live events to assist directors and production teams to potentially predict next frames and next moves for swifter edits, content detection and better workflows. The challenge will also aim to evaluate which datasets are needed to accurately predict live video outcomes and how far into the future the predictive video and audio can remain authentic to the actual video and audio.

ErinRose Widner, global head, business strategy emerging & creative technologies, media & entertainment, Verizon Business Group, explains how the project came about.

“We’re at such an interesting time in terms of technology and media, we’re entering into this new era of entertainment with the tools that are around us, whether that’s GPT or genAI. So one of the things that I was really keen to explore was around predictive; how predictive can we be in terms of immersive entertainment that we’re creating,” she says.

For Widner it’s also important to emphasise that this level of prediction means it is possible to personalise content while remaining authentic.

She continues: “There is this fear, as we step into this new era of media entertainment, that things are not going to feel authentic, and it’s just not going to be tailored for you. It’s just going to be these generic ads that pop up or things like that. So I really wanted to think about how can you create these experiences that feel more interactive and really immerse that viewer in whatever content they’re watching, so they feel like they’re watching it with someone who knows them. We’re doing an interactive conversation piece as well.”

While the tech could work across multiple genres of content, this proof of concept will focus on sport.

“I think sports is really interesting, because it’s live and so you’re getting to have that interaction about what’s happening live,” she explains.

Other participants in the project are Xansr Media, Magnifi and RKG along with Champion Paramount Global. Magnifi is sharing its expertise in the world of AI with the project team.

“The part that Magnifi is working on, and a couple of our other technology partners, which I think is really interesting, is that summarisation piece. So you can come into a game 20 minutes in, and basically hear a personalised summary of where we’re at. I think that’s very cool, because how often do you get to actually join a live game and get a summary of where you’re at in the game and it’s actually tailored for you. It focuses on the players that you like and the types of plays that you’d like to see happen, it brings you right into it, right from what you want.”

Geentanjali Dogra, events & content manager at Magnifi, says: “The pitch is all about personalisation of sports content and highlights and creating video highlights. Automating them using AI is exactly what Magnifi brings to the table.

“The concept was so interesting, because we already deliver on a lot of those skill sets, a lot of those requirements. But what this pitch does is basically stitch it all up together as a seamless journey. So you have a sport, you have the content, you of course make highlights, but today, you can hyper-personalise them and you can localise them. For example, I might want to follow a particular player, my friend may want to follow a separate player in the team. You both love the game but I might want to know more about a certain player, so fine tuning it to that extent. It was just a perfect fit for us to be a part of the Accelerator.”

Timelines for the Accelerator programme are notoriously tight, but the team is already making significant progress.

Nitesh Agrawal, senior VP of technology, Magnifi, reveals: “We have made a lot of progress across multiple fronts. We should have a working prototype in a few weeks, so it seems to be quite on track right now.”

While Verizon is something of an Accelerator veteran, this is the first time Magnifi has been involved in the programme, and it’s proving to be a great opportunity for learning and collaboration.

Agrawal adds: “We are collaborating very closely and speaking to each other every week. So, for example, working with Microsoft has been really interesting just to understand the insight, and to get access to newer models that they have built, which are not even released to the public yet. It is also a big learning for us in terms of how do you hold other people accountable when you have no control at all. So just by being super transparent, super collaborative and just being open in what we’re doing, this is what we need to do. Everyone in our group is very collaborative, it’s a fun work environment.”

Of course, the project hasn’t been without its challenges, however the biggest one has nothing to do with the technology and everything to do with the subject matter.

Widner explains: “One of the challenges that we had early on was, what is the content that we’re going to use for this project? We’re very conscientious of being ethical in the way that we use the content that we’re either deploying the solution on top of, or also training. And so one of the things was figuring out which content we could actually have. We went through the gamut of every different kind of sport and even thought about whether we could film our own stuff. So we had to figure out what the content was.”

So what can attendees at IBC expect to see?

Widner reveals: “If you come to the demo booth, you’ll have this personalised interaction with an AI that gets to know you and your preferences in terms of sports content and also gets to know what level of knowledge you have of a certain sport. So that way, they’re not just dropping you in if you’ve never seen something. So we’ll be able to gauge that, we’ll also be able to gauge some other interests, tone, things like that. And then once that’s been aligned, and we feel confident with what we’ve captured in terms of personalising that experience, then you will be dropped into the game. So let’s say that it is 20 minutes into the game, then it is going to show you a summary clip of what you’ve missed so far and then you get to watch the game as it goes with interactive commentary talking to you as you go. It’s going to be kind of cool.”

Agrawal is equally enthusiastic, concluding: “This can actually completely change the game and how we consume content, primarily on platforms such as mobile apps and websites. Right now with what’s being done everyone sees the same thing, All the OTTs, broadcasters they actually show you the same thing. But what we realised was that the current generation, Gen Z or millennials, do not have time to watch the entire match, so if you want to really engage them, you have to give them the content that they want to see.

“This entire system that we’re building makes it very easy for the broadcaster or an OTT to enable a very high amount of fan engagement on their platform. They’re relying on social for some of this, but this would actually bring all of that data back to the platform and enable them to offer a very personalised experience on their own platform. This has not happened before – it is very new for the market.

“This can really boost fan engagement, which everyone is after right now – everyone is asking how can we engage fans, how do we give them more, how do we make them come back again and again. For example, let’s say in cricket, a certain player is a part of the squad this week, you can send a notification, ‘Hey, your favourite player is going to be playing, watch the game’. This was not possible before. It’s so simple, but it has a very, very high impact on how you engage your fans.”

The IBC Accelerator final proof of concepts will be demonstrated at the dedicated Accelerator Zone during IBC2024 at the RAI Amsterdam (13-16 September) and also presented live on the Innovation Stage in Hall 3.