Endless AR possibilities: BT Sport’s Match Day Experience technical partners talk new realities
Here Match Day Experience on the BT Sport App is shown in Stadium Experience
BT Sport and mobile operator EE recently announced Match Day Experience, a set of new features for the BT Sport app.
Match Day Experience brings augmented reality (AR) elements and 360-degree video to live and non-live coverage of Premier League matches for EE 5G subscribers using the new iPhone 12.
SVG Europe caught up with Jamie Hindhaugh, chief operating officer at BT Sport, along with three of its technical partners behind some of the new features – Stats Perform, Second Spectrum and Visualise – to get more detail on how the features were created and where AR is going to take viewers in 2021.
Teamwork makes the dream work
Hindhaugh comments on how BT Sport has created the new functionality: “What I’m really proud of is the whole of BT Sport is taking people to the heart of the game and being editorially relevant, not gimmicky. The strength of being in sports is the community we build; it’s about strategic partners.”
He goes on: “I think it’s the combination of the products [in Match Day Experience] and really identifying from an editorially relevant point of view what football fans want, or what they can’t get in the current climate, that works here. But what we’ve also done is everything we’ve built is looking forward to the full 5G roll out and enabling fans to still engage with all of these editorially relevant products, no matter where they are. This is the start of, I think, an incredible journey for all of the teams combined to really transform the experience around watching live sport.”
On some of the companies involved in Match Day Experience, Paul Hunt is European executive director and European lead for football at Second Spectrum. In conjunction with the BT Sport team, Second Spectrum provides the video streams augmented with stats and graphics in Manager Mode for the BT Sport app.
Ross Tanner, Stats Perform account lead for BT Sport, helped provide the data-rich services across Opta to create visualisations for Match Day Experience; it provides data that powers the formations and attacking areas in Match Day Live’s AR features.
Meanwhile, Henry Stuart, CEO and co-founder of Visualise, an AR and virtual reality (VR) production agency, has taken the stats from Stats Perform and visualised them throughout the app, such as, “a stadium that you can place in your living room with all this live data”, according to Stewart. Visualise also created the app’s Stadium Experience, which enables the user to walk through an AR doorway as a portal into immersive content.
Other partners for the app are Sceenic, which supports the Watch Together functionality, Tiledmedia which supports the 360 functionality, and last but not least, Deltatre that integrates the Tiledmedia service into the BT Sport App enhanced player.
BT Sport’s new App functionality, Match Day Experience, here shown in Manger Mode
Endless possibilities for AR
As to how the combined technologies within Match Day Experience will help BT Sport’s viewers during the pandemic by allowing them to engage with sport that they are unable to attend in person, Hindhaugh says: “We’re taking people to the game, we’re giving them match day experiences they can’t currently get, and we are enabling them to socially come together. For me it’s about the experience.
“It’s about addressing the fact that in this current pandemic, a lot of us are on our own watching a game where usually we’re either at the game or we’re with mates watching the game in the pub or what have you,” Hindhaugh goes on. “So I think [the different aspects of Match Day Experience] complement each other tremendously.”
Stuart agrees: “The way we see it, this is just the foundation that allows us to build these first steps. You’ll see more and more impressive graphics and animations coming onto TV, either tracked onto pitches or in studios.”
Stuart notes that the possibilities are endless; through AR you can have, “a life size recreation of a player who stands there and you can have your photo taken next them to in your living room”. He adds: “Once the assets can be placed there, the possibilities are actually limitless. I think that’s what’s so exciting. And also the fact that BT Sport already produces so much live 360 content, as well as the prerecorded stuff that we’re currently using through the portal.
“You’ll soon be able to just be watching the TV [at home] then put an [AR] portal down and walk through to a live view, which is on the side of the pitch, you know, and then stand there and look around with your phone. It’s just an incredible way of watching the sport.”
Hunt says this is all pointing to a much more personalised experience: “I think that [this is about] making it more of a personalised experience. So rather being fed what’s been shown for the past 20 or 30 years on television, this is a new angle. It’s bringing new features and new excitement to it.
He adds: “We’re all different. We all want something different. We all want to see things at a different cadence. So I think this gives us now the opportunity to start looking at rather than just watching the full 90 minutes [even though] football is engaging – let’s face it, it’s been around for 400 years – it’s about delivering that next level, now. I think what we’re seeing here now will become very much part of the standard of future broadcasting.”
Hunt notes that the different functionalities of Match Day Experience also enable BT Sport to serve existing customers in new ways as well as attract and engage new or less dedicated fans. He explains: “It’s giving people something totally different, something new. The way we can use tracking data to automatically illuminate and elevate the game [means we can target] potentially a whole new audience.
“Maybe you’ve got some casual viewers and passionate fans; a casual viewer might be interested in the team he or she’s interested in, but not in the rest of the game, so [this app] can bring new dimensions to that, even something as simple as putting a name up above a head [of a player on the pitch] engages the viewer that little bit more, to want to know more. And that passionate fan, the fan who knows everything about the team but wants to see the game [from the perspective of] how the manager sees it, the strategy.”
Hunt adds: “Hopefully at some point we’re going to come out of this lockdown and [we can all get to] watch [live] sport. But I think all the experiences here will basically outlive [lockdown usage].”
Taking functionality to the broadcast
Hindhaugh agrees: “All of these [functionalities within the app] at the moment are consumer facing products. I can see, working with all of these partners, these [functionalities] becoming broadcast-facing tools to enhance our broadcast coverage as well, because every single one could to be used within our live coverage to give fans a totally different experience, which, again, is really exciting.”
He explains that a pundit could use the technology to give viewers a tour of the stadium, for match pre-plays or analysis. “I think these developments and how we can use them to up the experience is really interesting, for our pundits and for the people watching the broadcast feed.”
The app and Match Day Experience is set to be developed further throughout 2021, with a focus on fans within stadiums as they are allowed to return later next year. Hindhaugh says: “This is about continuously enhancing the experience. The one thing I’m really keen on, and this is where 5G is so important, is one sector of our audience; we really struggle to talk to other people who are actually at the games. I think if you can talk to those people, and if you can garner some of their experiences, you can use those experiences in a live environment to enhance your broadcast output because you [can] feel part of the conversation with the people in the stadium.”
Stuart notes that AR is gaining traction rapidly. He explains: “The fact that AR just relies on mobile phones has been the main reason it has picked up so much [this year]. I think VR is going to have some explosion at some point in the future when we’re able to actually film football and other sports volumetrically, which means that you’d be able to put a VR headset on and just walk onto the pitch, like an unseen ghost and be able to watch it from anywhere you want. And that would be enough to kind of get you past those social barriers of being isolated from the person next to you,” he notes, referring to the current antisocial nature of putting a large VR headset on, which is incompatible with the social nature of sports viewing. “It’s just going to be magical.”
Developing in the dark
The functionalities of Match Day Experience were developed last summer during lockdown, prior to the release of the device they are designed for, the new iPhone 12 on EE’s 5G network.
On the complexities of developing the areas that BT Sport’s partners were tasked with,
Stuart comments: “We started working on this when the new iPhone was still around the corner. It wasn’t there yet. And so we had to work on a lot of next generation of features, like the fact that the phone was coming out with a thing called a LIDAR scanner in it, which lets the phone understand the 3D world around it, [while making sure that] when the BT Sport [Match Day Live] app was launched, it had full capabilities to use that new device. That meant developing a bit in the blind.
Stuart adds: “Later into the project, before launch, [we had to make] sure that everything did work perfectly for that new device. So it was pressured in that sense. We also had to work with the existing BT Sport app developers to try and integrate an incredibly large and complex app already, in with our parts, so they could pull this data in as and when it was needed, or so it could pull in the latest 360 videos in a very seamless way that was easy to use for the team. So technologically, it was just quite a difficult challenge to get it all going in time.”
Hindhaugh says that the app’s UX was also important in the development process: “Not only was Stewart and the team building a new app, but they also had to deal with my creative and design staff around the UX and try to understand that some of the foibles absolutely can change how you build something from a technology perspective. I think that was quite a challenge at first, because there’s different ways of showing things, [but] it was great how the teams pulled together to make that happen.”
Intuitive, realistic and seemingly simple
Hindhaugh notes that the hard work behind the scenes has created intuitive, realistic and seemingly simple functionality. “The doorway [for Stadium Experience], for example, looks incredibly simple. It looks very natural. It’s got these transparent glass doors covered in the logo of the team that’s playing, and those pull apart to reveal the match behind, or reveal the tunnel [in the stadium] or something like that. It feels very natural and you just walk through; it’s this amazing moment as you cross the threshold, but actually there’s a huge amount of tech going on hidden behind the scenes.”
Back to the development process, and Hunt agrees that the timeframe was a challenge. “We started talking to Jamie and the team probably in the summer, so time was of the essence. That was the main thing in terms of two teams trying to work together, trying to understand how each other works, but fortunately [we had the same] sort of vision; that this is how we see the future of sports broadcasting, and how we want the [app] to look. The way the two teams merged and worked together was nothing short of impressive.”
Meanwhile Tanner’s team had to establish what sort of metrics the BT Sport team required as part of this solution. He says that establishing the, “storytelling metrics that we wanted to provide through this solution,” was the challenge.
“But for us, we really are only touching the surface in terms of what data metrics we can put within this [app]. I think there’s so much more that we can do in terms of enriching the experience,” Tanner says.
Hindhaugh concludes: “The combination of all of these evolving those products – and, you know, the AR portal, when I’m sat eating my hotdog at the match, reading the programme, overlaying AR onto the pitch in front of me and to being able to go around and look into areas that I can’t actually look at, even if I’m allowed to go to a game – I think is phenomenal; really exciting.”