Nordic Summit: Clark, Hellmuth, Horneland and Jakobsen discuss graphics, AR and AI
Clever creation tools, graphics augmentation and increasingly sophisticated analysis tools are very much at the centre of new developments in sports broadcasting, not least to engage and retain those less-than-dedicated millennial media consumers.
‘Graphics Present and Future’ panel at Media City Bergen, January 17: (L/R) Kjetil Horneland, Petter Ole Jakobsen, Steve Hellmuth and Doug Clark
At SVG Europe’s first Nordic Summit, held in partnership with Media City Bergen and NCE Media, Sixty CEO Kjetil Horneland led a panel featuring Doug Clark, IBM Global Cloud & Cognitive Solutions Leader; Vizrt CTO Petter Ole Jakobsen; and Steve Hellmuth, EVP Operations & Technology at the NBA in a discussion about the present and future of this important market segment.
Horneland began by asking Vizrt’s Jakobsen how he sees the broadcast industry changing in the area of sports. What are the biggest challenges ahead? “There are some super-simple trends,” Jakobsen replied. “One is mobile, obviously. Another one is screens, and increased resolution. A third one is AR. And a fourth might be physical things: all this sport has to be a package of entertainment, with something for everybody.
“And whether you end up using visual techniques or other techniques to achieve that, you’re doing it for the same reason. We are so blessed that as long as I’ve been in this industry we’ve been able to continuously produce more advanced and realistic graphics, plus cheaper and and screens – and that trend is just continuing.
“There was a recent study into media companies by Columbia University,” said Jakobsen, “and it showed that some of those organisations are now publishing to 20 different platforms. So if you have some content, and you know that your viewers are fragmented in terms of consumption, you really have an issue there. One of the challenges is to find methods to ensure that enriched content – with graphics I mean – can reach all these devices in an efficient way.”
How the NBA engages with fans across mobile and social
Steve Hellmuth had previously conducted a one-on-one interview with SVG Editorial Director Ken Kerschbaumer earlier in the day, going in-depth into the way the NBA engages with its fans on-court, on television and on mobile devices around the world. Panel moderator Horneland picked up on this interview, asking Hellmuth to talk a little bit more about how the NBA engages with basketball fans across mobile and social media. The NBA has been innovative in many of these areas, observed Horneland, asking Hellmuth to explain his thoughts on how to retain users as we move ahead into the future.
“We have an engine where we are rapidly logging, tagging and capturing the video and it goes to editors who sit on top of different social media channels,” said Hellmuth. “They do what’s then appropriate for that channel – you don’t want to kill someone with 30 clips.
View of the atrium at Media City Bergen
“There’s a level of appropriateness with Instagram that’s different to WhatsApp and different to Facebook. And on top of that we actually use a Viz tool, Liberovision, to provide a layer of graphics. Any time people see video that includes graphics, analytics, includes some insight along with a great story and a great series of plays – that’s what attracts them. That’s what takes it from 200,000 hits to 2 million, right out of the box,” he said.
“They are learning more, in less time. The difficulty for us really is in retaining people at the NBA who have this kind of training, and really know basketball – people who can make the subtler aspects of the game apparent immediately.
“What we’re doing is working with Petter Ole and his team to have the player tracking data interfaced to the Libero so we can produce things rapidly, and get stuff into the first replay in the telecast,” said Hellmuth.
How AI is boosting the impact of tennis Grand Slams
Turning to IBM’s Doug Clark, moderator Kjetil Horneland said, “We’re moving into an abundance of data. Everything can be logged and everything is becoming more and more available. It leads to a point where we will still be telling some stories manually: but when we come to data and AI, there’s a lot of talk about what’s going on but we don’t have many examples of good case studies out there. Doug, do you have some good examples of what IBM has been doing in AI in sports?”
“Sure,” said Clark. “First of all, thank you very much for today – it has been an exceptional day. From an IBM point of view we’ve been marrying three separate components. First of all we’ve investing in cloud for quite a long time. We have also acquired over the last few years some cloud video assets, live streaming and OTT. And we’ve been working for a long time not just in analytics but moving that on to machine learning – basically what we’re all now calling AI.
“Marrying those three things together, the acceleration of the innovation has just been astonishing,” said Clark. “One of the best examples I’ve got is tennis. IBM has been blessed to work with most of the Grand Slams over a good number of years: I think last year was our 27th with Wimbledon.
“I remember the first time I was there, we had guys running film from the on-court still cameramen to motorbikes to be transported to central London to get to the newspapers. Then it became digital; and now it’s all totally wifi and totally interactive live. That whole evolution has gone really fast.
“And with Wimbledon it’s a really historic, sensitive environment,” he said. “The All-England Club love technology – but it has to be hidden. It’s still etiquette and blazers and strawberries and cream.
“It used to be we would put a proposition to Wimbledon and they would say, “Yeah OK, maybe we’ll trial it this year and if we like it we might think about it for next year. Now it has gone beyond that: maybe we do something for Roland Garros and then we go big bang for Wimbledon, which is only a few weeks after. Straight into the big bang.
“If you think about that acceleration of innovation: what we did last year for the first time at Wimbledon was pretty much live highlight feeds using Watson technology for one match. That was in June. By the time we got to Flushing Meadows for the US Open, we were tracking seven matches in parallel – and being able to do live highlights from those seven matches.
“If you put that into context for the tournament,” said Clark, “you’re talking about 17 matches simultaneously; six matches per court at any one time; 25,000 points to judge and identify for interest; and over 300 hours of footage. It’s a lot of stuff to assimilate and judge, from an AI point of view.
“First of all with Watson Media we do the transcription of the commentary. You can tell a lot from the commentary. Then you’ve got the natural language of the commentary, including the tonality and the emotion. Then you can take the crowd noise – which is different for every match and in every stadium. Then we do facial recognition, so you can see the emotions and the gestures. You can teach Watson to understand the game of tennis – it’s sensitised to the crucial points of the game.
“If you put all that together you can score all of those points, and then that creates an excitement rating and then you can put that into a league table,” said Clark. “And as the games are coming in live, those points are accumulating or moving you up and down. If you only have time for five highlights you can put the top five in and so on. That’s how that highlight reel is generated.
“How do you understand which points were really good?” he asked. “Then think about the camera angles – you can train it [the AI] to recognise that one camera angle is the service and another is the conclusion of the point. So you can put what you can describe as a ‘scene’; each point becomes a scene, and then you can cut and paste those scenes together to create the highlight reel,” said Clark.
Personalised and navigable graphics for NBA fans, with Sixty
In concluding the session the NBA’s Steve Hellmuth doubled back to talk about the work the league is engaged in with Norwegian company Sixty. “I would be remiss if I didn’t mention the work we are doing with Sixty, right here in Bergen. Obviously when you get people to your platform, the trick for any sports league is to just hold them longer. If you hold them longer your ratings go up, your fans are happy with your app and your advertisers are happy too with the growing audience. That’s really where we monetise,” said Hellmuth.
“So what we’re working on, particularly with the Mobile View feed, is how things are changing –
Steve Hellmuth: “We’re working with Sixty to make it navigable”
second screen is now first screen. I’ve designed so many things so many interactive things for PCs over the years … that nobody interacted with! They were leaning back from their PC; their hands weren’t there.
“Now that the mobile phone is the first screen for many, 23% of the people watch the NBA – our League Pass audience, which is about 400,000 strong around the world – on the mobile screen. And that’s growing steeply. Their hands are right there.
“In terms of augmentation we’ve worked with our statistical feeds and our clocks, in order to allow fans define what graphics they want to see and when they want to see them. We’re working with Sixty to make it navigable, so that you can literally live on this device and see what you want to see because your fingers are right there.
“I think it’s really important work, and the delay in the encoding allows us to store graphics to the cloud and deliver them synchronously with time code. We really look forward to doing more,” said Hellmuth.