German milestone: Sky Deutschland creates the first live remote production of Dolby Atmos for a major host feed with the Bundesliga
Sky Deutschland has created the first remote production of Dolby Atmos for a top-flight sport’s live major host feed in Germany.
At the Deutsche Fußball Liga (DFL) Supercup on 30 July where RB Leipzig faced off against Bayern, Sky Germany provided fans with full immersive sound, working with the DFL’s technical services provider, Sportcast, which deals with the on-site production and technical set up for the Bundesliga, its partner, Vidi, and audio expert Professor Felix Krückels.
In just four weeks, Krückels and Vidi developed a Remote Audio Control Room (RACR) based in Darmstadt, and have dealt with a complex array of issues along the way to make Dolby Atmos a regular feature for avid fans of German football.
“I can tell you it feels with the fast network for all control data (surface to control system to UHD Core and monitoring back to Darmstadt) like I am sitting in the truck onsite”
Krückels, who is Professor for Broadcast Production and System Design at the University of Darmstadt, Media Campus Dieburg, along with Vidi were tasked with deploying the RACR for the production of the international host feed in 5.1.4 plus six objects, and in parallel, the 5.1 and stereo downmix to generate the international sound, as well as the Sky Germany mix for the match in 5.1.4 plus two objects.
While the first match to showcase the remote production of Dolby Atmos in a top tier sport was the DFL Supercup two weeks ago, it is now being used on all subsequent top matches, which in Germany are regularly played during the season at around 6.30pm on a Saturday.
Now is the time
As to why Sky Germany decided this was the right time for Dolby Atmos to go remote on live sport, Alexander Lutz, head of next gen production platforms and content technologies and platforms at Sky Deutschland, told SVG Europe: “Sky Germany, in agreement with our service provider, decided to do remote production because there’s simply no space left at most of the stadiums to get another OB van next to the existing [broadcast production] one, which is required to do a proper Dolby Atmos mix. If you can imagine, you need a proper calibrated, integrated acoustic environment, and this is not within the normal UHD OB van.”
Lutz continues: “To produce in Atmos, you need an extra silent and calibrated environment for all the 3D speaker positions. Either you bring another OB van next to the main production truck, and at most of the [Bundesliga] stadiums there’s no space for that, or from the other side, it’s also not optimum because even within the OB truck, it’s not on a permanent environment for advanced audio. So remote production means we have a dedicated room somewhere. It’s just the one for all these productions; it doesn’t have to be transported somewhere, it’s always measured and calibrated and you can produce all events from one single place.
“This was the solution for us also because the technological solutions are already existing. Especially for audio signals, the latency is very acceptable if you do a remote production because the network is fast enough and the bandwidth within Germany is OK, so you can have a proper network connection to the remote production,” concludes Lutz.
Adds Tim Achberger, director of technology and product management, Sportcast: “Our goal was to develop a remote-based immersive audio production concept which is scalable for our various national and international licensing partners.”
The challenges in this deployment were multiple, says Krückels, the brain behind the system: “The technical challenge was to minimise latency to not force the video path into adding [any extra] video delay. You have to remember that the remote mix is also the international sound, which is used for all HD feeds, all ISO recordings to EVS machines, etc. That means if the audio would imply more delay than the cameras do anyway, it would have been necessary to delay all cameras and all feeds, and we are talking about 25-plus cameras!
“Also, network latency changes from stadium to stadium, which means we knew the audio and video difference would have been different from one stadium to another. Therefore we proposed a solution where the control room is in a central location, in Darmstadt, in a controlled environment with proper speaker set up and the space for it, but the mixing console DSP is on site in a small stage box; the RACR stage box. This solution prevents [the system] from adding on roundtrip time. In Darmstadt I am receiving only the monitoring output for speakers and headphones, 10 channels and a bit more for headphones and metering; it’s like I am mixing onsite in the stadium, but I’m actually sitting in Darmstadt.
“The second request was that we are not yet able to synchronise all OB vans and Darmstadt to the same clock, in this case PTP. Therefore we simply added MADI SRC’s between the truck and the RACR Stagebox.”
“It’s not simple to produce Dolby Atmos. It’s really not simple. Dolby was involved from the beginning [of planning this with Sky Germany] and had to deal with a lot of issues and solve them and consult us. It is not easy to manage those Dolby audio channels on top of the existing infrastructure”
Krückels and the Vidi team deployed a complete RACR with a 9.1 monitoring set up. The heart of the system is a Lawo mc²56 MK3 with two UHD Cores and a PowerCore attached to it. All systems are controlled by Lawo Home for Stream set up, as well as redundancy scenarios.
Notes Krückels: “In a live situation you always need redundancy for your main mixing device, which in this case is the UHD Core. As the UHD Core is a 100% IP core, it does not matter where the core sits. To keep the RACR Stagebox small and the redundant system reliable, we decided to put the redundant core in Darmstadt to the Vidi server room. This means, if the UHD core on site would fail, the core in Darmstadt would take over. Certainly in that scenario we would incur the roundtrip time as additional latency, but that was acceptable given the very unlikely event of a failure.
“Also all three Home servers, housing Home as well as the main mixing control system in a redundant set up, are placed in the server room in Darmstadt. That means we could minimise the Stagebox set up to just four RU – one for the UHD Core, one for PowerCore, one for the switch and one for a PC for remote maintenance plus the video and network interface, a Media Links MDP 3020 unit. I am receiving a multiviewer with 10 PIPs in the control room, encoded and decoded with JPEG XS, which minimises video latency dramatically.
“I can tell you it feels with the fast network for all control data (surface to control system to UHD Core and monitoring back to Darmstadt) like I am sitting in the truck onsite,” Krückels adds.
Getting it together
The system was tested in a proof of concept back in February this year using a rental Lawo system. Says Krückels: “It was prepared in a way that I could go on air already in February. In fact we already used the Kick mix (the close ball mix automatically controlled by Kick software with data from the players and the ball) during the POC, which means I mixed the close balls with Kick and sent back the close ball group to the sound engineer on site.”
He continues: “The biggest challenge is to get it all together: the all-IP set up, PTP scenarios for both locations, all presentations and downmixes for both feeds for the DFL and Sky. You have to keep in mind that each presentation requires at least two downmixes – surround and stereo – which means you can easily end up in 12-plus scenarios you have to monitor and check all the time. I am used to that scenario already from the last two football tournaments [using Dolby Atmos on this system], but still it requires a lot of preparation and thinking through it.”
For the DFL and Sky Germany, Krückels decided to provide the downmixes the same way as he would set up presentation. “Right now in Dolby you can only attenuate the height and surround channels, which limits the possibilities a little bit. For football in 3D we tend to use the height channels not just for crowd or reflections, but also to content-relevant audio such as the PA. Similar to the best seat in the stadium, where the PA is reaching you from above or even from upper rear, we put the PA feed to the height channels too.
“If the downmix parameters of these signals is too high, the surround and stereo listeners would lose that content. That’s the reason why we decided for only -3dB downmix of the height channels. Overall that gets a bit louder compared to the discrete 5.1 mix, but it works very well with the stuff you want to provide to your customers.
“The second big challenge was time,” continues Krückels. “We had just five weeks after the initial decision for Vidi and me to set up the complete control room and the Stagebox. Due to the successful POC we could reuse a lot of the planning details from the first scenario.
“Another challenge for Sky was to use the second quadrant through the complete signal path. Very early on we all decided to not use ED2 for contribution due to the high latency that implies and the complexity it brings with it. Luckily Sky could manage that, and we can now use 32 audio channels in the UHD feed, which simplifies the workflow a lot.”
Adds Lutz: “It’s not simple to produce Dolby Atmos. It’s really not simple. Dolby was involved from the beginning [of planning this with Sky Germany] and had to deal with a lot of issues and solve them and consult us. It is not easy to manage those Dolby audio channels on top of the existing infrastructure.
“For example, we had to carry all the Dolby stuff on the second quadrant within our UHD contribution because there’s just simply no space for audio within the first quadrant, which is fair definition for UHD if you do a quadrant-based production.” Lutz continues: “So all the audio should be within the first quadrant, but there was just simply no space for it. And they had to use the second audio quadrant to contribute 16 channels of discrete audio for the Dolby Atmos production.”
Says Krückels on the overall challenges and what has been learned so far: “It’s always a challenge to set up a complete new system. We had some timing issues – a lot more audio delay than we expected – but we learned that with a specific boot-up procedure, the system synced to each other in ways that mean we now reach the expected latency between the OB van and the RACR Stagebox.
“Audio mixing-wise it is also always a challenge to start with the new mixing console set up. Although I know the parameters for a proper sports show inside out, you still have to put all of that together. And that for the first match, we had the German Supercup!”
The DFL’s broadcast arm, Sportcast, added a new fan microphone for the Supercup on the left and right stand, a Schoeps ORTF 2+2. Says Krückels: “This was a system I evaluated in pre-COVID times at the university with my students, with very good results. The microphone delivers very good localisation and immersion, but again the set up and position of the microphone is key for good sound.
“Due to the distance between the stadium and the RACR now, I can’t walk to the pitch and verify the position [of those new microphones] by myself, so that requires a lot more communication and explanation.” However, he adds: “TVN, providing the OB trucks on site, prepared the communication path in a very cool way so that I can talk to the audio technician the same way as I would if I was sitting in the truck itself.
“The final step perhaps would be to put my eyes on the audio technicians head with a GoPro or similar,” muses Krückels. “I am still convinced that 80% of good sound is coming from the right mic position with the right microphone!”
As to what has been learned so far at Sky Germany, Lutz states: “Finally, it’s worked, like everything has worked and it’s working right now almost as expected. But just the feed – just the football game itself – is in native Atmos, but the rest of our programme on the channel is still 5.1 or stereo and all of that has to be aligned by a special logic. And we still have even some special form of commercials, for example, where we have some graphical overlays with some commercials on top of the live feed and those scenarios are hard to power, so we require some special functionality [to make it work]. To get this done as well, to be able to combine all of that, is complicated.”
Lutz says that at the moment, Sky Germany is limited to doing the remote production of Dolby Atmos on Bundesliga only as things that are simple in a 5.1 production, such as taking an international sport feed and putting a German Commentator into the mix, is significantly more complex in an Atmos environment. “This is not that trivial and not that simple in a Dolby Atmos world, because those are objects, which have to be exchanged, and there is metadata, which has to be handled, and this is not a simple thing.”
Krückels agrees. A challenge in 3D audio is to get the commentary, sitting on the side of the stadium, cleaned up from all the audio spill in the stadium. Krückels says German commentators do not like handheld mics, which would be an ideal solution to solve this problem, instead preferring headsets.
He comments on the issues this creates: “Quite often the headset mic is not in the right position. All this spill is even more obvious in 3D audio due to the cocktail party effect. In surround, and even more in stereo, that is quite nicely masked by the crowd sound. But as we have two commentators with open mics all the time for Sky, it’s even more problematic. I am already using automix, expanders and a special trick with the automix to reduce the spill, which works quite nicely, but it also takes a lot of effort and time to get it right.”
Krückels concludes: “We have to improve the commentary sound with whatever it is possible to use – artificial intelligence for spill reduction, microphones with less spill sensitivity, and anything else – to get a much better com-spill ratio but that feed needs to be cleaned up a lot more than we are able to do today.”