Sky Sports on “smashing out” virtual Premier League audio with Dolby Atmos and Ableton Live
Sky Sports sound supervisors bought Premier League matches to life using DJ application, Ableton Live
Sky Sports has used Dolby Atmos throughout Project Restart, which has marked the return of football to the screens of those in lockdown.
The broadcaster brought the Premier League and the English Football League (EFL) to life for fans at home, in a process that was both fast, innovative, and not a little risk taking.
The use of audio at football matches played in empty stadiums has been a contentious area of conversation in socially distanced pub gardens throughout Europe since football came back. However, Sky Sports has made it work.
The broadcaster began working with Dolby Atmos in 2017 for the Premier League, and now that relationship has grown to encompass the EFL, Autumn International Rugby, pay-per-view boxing, to the World Darts Championship, and out into the world of entertainment.
Jane Gillard, senior broadcast manager at Dolby Europe, noted that when Dolby first began working with Sky Sports on Dolby Atmos a, “critical component of the Dolby Atmos experience in particular for live sports was crowd sounds”.
Gillard went on: “We quickly realised that our ability to convey crowd ambiance immersivity in Dolby Atmos would give us an opportunity to make fans feel like they’re in a football stadium, like never before. We could use the technology to convey the sounds of a particular stadium because many stadiums have their own sort of unique sounds. We could make the team chants really stand out and be conveyed a lot more clearly. And also we could really bring those goal celebrations to life.
“Really throughout our testing and also throughout the live broadcasts with Sky I think we’ve shown that that crowd sounds are really a core part of an immersive sports experience,” she said.
Close up of the myriad of buttons and faders used by Sky Sports sound supervisors
Realistic virtual audio
Yet while the pandemic put a swift stop to plans for both Dolby and Sky Sports earlier this year, Dolby Atmos has been a key part of Sky Sports’ Premier League restart plans to bring realistic virtual audio representing fans to viewers at home, in the absence of a crowd at the stadium.
Kevin McCue, director of technical operations at Sky Sports, spoke about how audio became a vital part of the Sky Sports Premier League offering over recent months.
The process began when McCue and James Clement, director of operations at Sky Sports, got together to work out what would be possible when live football returned, in terms of enhancing the experience for viewers who would be looking at empty stadiums.
Said McCue: “It was crazy actually because a lot of companies approached us about the concept of [creating virtual] live crowds in stadiums when there’s no live crowds there. And a lot of the ideas out there were actually about playing crowd [sound] live inside the stadium. So actually it was [about] big PA arrays and playing that back inside the stadium and the whole kind of fan engagement thing where potentially [the solution] could be [based on smartphone] apps and people could shout at their phone and it would come out in the stadium, and all this other kind of good stuff.”
However, while a lot of companies had great ideas, none could promise the broadcaster that, “we wouldn’t get any profanity,” said McCue.
“From a live broadcast perspective [profanity broadcast live] is obviously not going to be acceptable,” he went on. “We just felt very uncomfortable with this idea of essentially unleashing a group of people on our live broadcast that we have absolutely no control over and anything could happen. And for various reasons, including things like transmission delays and all this other kind of stuff, it just really [was not going to] work. We pivoted on that on that kind of thought process and reached out to our colleagues at EA Games in Vancouver.”
Sound supervisor operating Sky Sports crowds for the Premier League at Sky Studios
Sky Sports has had a good relationship with EA Sports for a number of years; it is an extension of the relationship the broadcaster has with the Premier League, whereby Sky Sports records a lot of the Leagues crowd sounds on its behalf.
“So at events, we will multi-track record stuff for them and then hand that over to those guys [at EA Games] and they very meticulously cut through and categorise and curate these sounds for the FIFA football game,” explained McCue.
McCue said that when he spoke to EA, they were expecting his call. “I knew they had to have really good ideas about an approach that we could use to play crowd back. They had had similar thoughts to us on how could we do it and how we could actually trigger an event and how we could create a dynamic cloud environment.
“We had thought about using AI and computer vision and various other things using crowd interaction or viewer interaction, and quickly discounted a lot of these because from a latency perspective, the viewer at home would be late; they would not be reacting very quickly. AI couldn’t really react quickly enough either from a computer vision perspective.
Hey Mr DJ
“So we both kind of come to the similar idea separately; it would involve someone actually playing the crowd, so this concept of a DJ playing the crowd, which is pretty much where we ended up. So we took away [EA’s] offering and presented that to the senior guys within Sky Sports, so the likes of Steve Smith [executive director of content], and Gary Hughes [head of football] who looks after the Premier League.”
“They literally just sit there, moving a fader, pressing a button, and they got to the point with it that they could start to almost predict what’s going to happen”
McCue said Smith and Hughes were initially slightly dubious about the proposed DJ solution, but after more demonstrations and opportunities to explore the way the crowd sounds could be manipulated and curated, they came around to the idea.
He noted: “They very quickly thought, “you know what, actually, this is good thing”, and quickly pivoted back to us and said, “right, we want to do it on everything, on all of our Premier League fixtures when they come back and you know what, we want to do on the English Football League as well”. So it very quickly became a little bit of a tumbling snowball.”
Next up came staff training. From a standing start of doing some basic demos, to working with EA, to working with a partner EA has in Vancouver, Canada, that produces soundscapes and other assets for them, the project advanced swiftly.
Commented McCue: “We had to give the guys [at EA] a whole set of fixtures. They were producing elements for us within an application called Ableton Live, which is a DJ application, and sending us Ableton Live sessions.”
EA was able to clean a loop so well that the human brain was unable to detect any repeating sounds to recognise the fact that it was listening to a loop, said, McCue, “so you can play a loop of crowd and you wouldn’t know it’s a loop; you just think that it’s this kind of constant thing, you don’t hear the loop in it. And [EA] would have removed things like whistles and ball kicks and profanity and shouts and screams, and curated that into a very cleansed archive of sounds that we could use confidently.
“Then we had to kind of face this idea that we’ve got the technology, we know how we’re going to play it back, we know how we’re going to put it into the [live football] programming. Now we’ve got to train a whole bunch of people that maybe have no experience of Ableton and no experience of playing a crowd live and no way really to rehearse it, [but we have to] train all these people and go live with it. Which is essentially what we did; the first game of the season we went live across the Premier League and it’s gone from there, and it’s gradually got better over time.”
“Walk this way” at Sky Sports Studios
Smashing sound supervisors
He went on: “The success of this at Sky is about the people doing it. We had people that were in booths, just smashing their way through this stuff all day and thinking about it, and working as a unit and enjoying it and taking the positives with the negatives; taking the excitement of getting a really good game with the flip side of making a mistake.
“COVID has made people a lot more pragmatic; that’s certainly the thing that we’ve experienced with some of the things that we’ve done over the last, three or four months. Steve Smith from the outset was very much, “Guys, you’re going to make a mistake – that’s okay – let’s just try to do the best job we can”. When you’ve got those kind of words that are wrapped around something like this, people are willing to try and push the envelope a little bit. And it’s a credit to the team, and it’s a credit to one particular individual, a guy called Adam Perry.”
McCue said sound supervisor at Sky Production Services, Perry, “kind of came charging in,” as a member of the Sky Sports sound team early on in the process. “He said, “I know how to use Ableton, and I’ve done a bit of DJing. Let’s have a look at this”, stated McCue.
“It very, very quickly went from something that was quite flat as a stereo proposition, to something that was very immersive and completely wrapped around you in an Atmos proposition”
He continued: “He spent the first week at home with a laptop that we found with Ableton on it and a mini keyboard, and at the end of that period of time, he’s like, “right, I need a mixer. I need some buttons. I need this, this, this, this, and this. I think I can make this work”.
“We supplied him with all the kit and off he went. He did the first couple of [live football] events himself, and it was then, “We need more, we need more, we need EFL, we need this, we need that,” to the point where we had on the last day of the season eight people doing Sky Sports crowds at the same time, and one of them had only been trained for a week! So that poor person in the booth just sitting there, desperate not to make a mistake.”
“Move a fader, press a button”
The way the sound supervisors were presented with the tools of their trade on Ableton was a constant anticipation loop of general crowd sound, and then a series of buttons linked to samples of chants, songs and cheers – specific to each team – as well as samples of boos, hisses, whistles and reactions to specific events on the pitch, all linked to volume control. In total, each sound supervisor had around 16 faders and 128 buttons to work with.
Commented McCue: “They literally just sit there, moving a fader, pressing a button, and they got to the point with it that they could start to almost predict what’s going to happen [on the pitch as they watched the live game].”
Socially distanced double match galleries at Sky Sports Studios
The sound supervisors were based in the international commentary booths at Sky Studios. Said McCue: “Putting all the excitement of this to one side, our whole thing through the whole of lockdown is been about keeping people safe and actually not unnecessarily moving people around, not unnecessarily making people work too closely to each other. And there was a good few months in lockdown where we were making most of our content actually in the cloud with people working from home and using mobile phones and iPads and their laptops; we were genuinely producing probably about 300 to 500 hours of live content with people just sitting at their desk at home, making this stuff happen.
“So the international commentary [area] was a good spot [to host the sound supervisors] because we have all these booths, they’ve got sliding doors, we can put a person in a booth, we can give them all the kit they need, we can make sure that the same person goes into same booth, we can wipe the area down every night so we can keep it nice and clean and hygienic and safe.
“What that does mean is that they’re there to support each other. So they can kind of shout from one booth to another with their door cracked open, “how would you do this?” or “how do you do that?” or “how do we develop this?” And they’ve worked together as a team like that, but all in isolation, the isolation of a [voice over] booth.”
On the team that pulled this off, McCue said: “What a brilliant opportunity to do something that’s never been done before like this at scale on such a huge [broadcasting] right as well… The Premier League is like our crown jewels. We’re very protective over the Premier League, so for sound supervisors or junior sound supervisors to literally be right there in the production, like right at the front of it is… I mean, on the last day of the season they were kind of like a socially distanced huddle of a sound supervisors, all just looking back at what they’d actually done. It was a really fulfilling experience.”
Immersive wrap around
Sky Sports did not originally start out using Dolby Atmos, but a senior sound supervisor tried it out. McCue commented: “We listened and it sounded brilliant! The sound supervisor asked, “when will we go live?” and I said, “just go live with it. It’s as simple as that. We’ll monitor it, we’ll make sure we’re okay. Just go live with it”.”
On early challenges during the live broadcasts, McCue said there were some yet the learning curve was fast. He explained: “Initially there was a few mistakes; I think we triggered a goal when it was a miss, and various things have gone on, but over time it become very refined and actually became a really integral part of our broadcasting.
“What the guys from EA presented to us was a set of [Dolby Atmos] stereo samples that we could use live in a game. What we wanted to do is really kind of granulise and control the samples, so using Ableton live and using some of the functionality of Ableton Live we could segregate different sets and different types of sounds and group those together, and actually send those into the live production individually.
“So essentially, what could have been delivered as stereo, we ended up delivering a kind of a 4.0, so a surround sound on four corners, and then we also added some height elements and all of these triggers and sounds could be triggered separately.
“It very, very quickly went from something that was quite flat as a stereo proposition, to something that was very immersive and completely wrapped around you in an Atmos proposition. It was almost a no brainer to us to try and be able to offer the most kind of compelling and comprehensive kind of Sky crowds offering.”
Using some enthusiastic members of Sky Sports’ inhouse sound team and taking the time to think about what was being created, McCue concluded, “I think we actually created a pretty rich soundscape for the Premier League”.