Broadcast evolution: Formula E technology director Eric Ernst on taking the all-electric motorsport into a new centralised remote production

This year in Season 9, 22 drivers from 11 teams including iconic racing brands McLaren, Porsche, Jaguar, Maserati and Nissan lined up on the grid of the most competitive and entertaining season of Formula E yet

Formula E has come a long way since its inaugural race in Beijing back in 2014. The world’s first all-electric motorsport world championship mandated by the FIA – and only sport to be net zero carbon since its inception – Formula E leads the elite sports industry in showcasing how high performance and sustainability can co-exist.

This year in Season 9, 22 drivers from 11 teams including iconic racing brands McLaren, Porsche, Jaguar, Maserati and Nissan lined up on the grid of the most competitive and entertaining season yet. With such an elite racing product on the track, it was critical that the broadcast product also took a huge step forwards.

Starting this season, Formula E’s most significant gamechanger was to take the host broadcast technical facilities of all its content inhouse as a managed contract. This is based on a new broadcast system developed by the Formula E Technology Department and managed by Eric Ernst, technology director at Formula E.

Speaking to SVG Europe, Ernst comments on this evolution: “Formula E is a very young sport and the goal at the beginning was just to get something going as soon as possible to prove the concept of an all-electric motorsport. The founders back then just went to a whole bunch of their contacts and said, “Can you do TV? Can you do timing? Can you do whatever? Just do it”.

“This is not a broadcasting department. This is the technology department stepping in and saying, “this is basically a networking operation and we can run this like we run the rights holder distribution or the timing network, like we run the connectivity into track; we can also engineer, provide and manage these services””

“The product has now outgrown that [original] workflow. When I joined five years ago, coming from the Volvo Ocean Race where we were already doing remote production on a smaller scale, I looked at the whole thing [as it stood then] and said, “this is probably not the direction we should be headed”.”

Ernst wanted to push the ABB FIA Formula E World Championship further along to maintain its sustainability credentials, as well as making the entire system that the broadcast is based upon more reliable, flexible and futureproof.

The Gravity Media Production Centre for Formula E at White City, London

Overriding escalating risk

The pre-existing system was based on ageing broadcast equipment being sustained to maximise the investment. However, the issues and outages that began arising were escalating in risk, according to Ernst.

He notes: “There was a need that we really, really had to do something soon, change the model and look at what’s available in the market today. I have a lot of experience in building enterprise networks, enterprise deployments and doing global sporting events for the last 15 years.”

What happened next is something of a coup, where Ernst’s team in the technology department were entrusted with the technical facilities of the host broadcast for the championship. Ernst explains: “It took a good three or four years for us as a company to get to a maturity level where it entrusted us to take this over; and I’m talking about the technology department. This is not a broadcasting department. This is the technology department stepping in and saying, “this is basically a networking operation and we can run this like we run the rights holder distribution or the timing network, like we run the connectivity into track; we can also engineer, provide and manage these services”.

“That’s where we [the technology department] engaged. We have massively grown as a championship and there were areas that we could run more sustainably and others where we could optimise the technology.

“We initially did an RFI, then companies who responded to the RFI went into an RFP process. After selecting our host broadcast partner we took over the management last year, well, this season, and we have done eleven races in this set up to date.”

Centralised remote production

The new set up is based on a centralised remote production model with a small footprint on location, and around 50 staff that were previously on site are now working from the UK at Gravity Media’s Westworks production centre in London, including people on racking, tv graphics and audio.

Connectivity for the broadcast is over the public internet rather than dark fibre, facilitated by Tata Communications. “We operate a data centre with really high availability out of a parking lot,” notes Ernst.

“There was a lot of convincing that had to happen, because everybody’s scared when something goes black, for obvious reasons. It’s a hard way for a supplier to prove to you why it’s not their fault or it’s a hard way for a client to prove why it’s the supplier’s fault. Taking down this mentality and getting rid of the finger pointing mentality of just kind of opened up this playing field”

Regarding the connectivity over the public internet, he adds: “My background in networks taught me that if I order a certain amount of connectivity, I pretty much get almost a similar service as when I get a dark fibre. The networks are now good enough that the added protection I would get with a dark fibre I can mitigate with a second fully diverse line.

“With our partner, Tata Communications, they can move worlds for us. We’re sending around 90 feeds back to Westworks over a 2GB internet connection. At Westworks we have our studio, so we’re putting that up and then sending a show back with a glass-to-glass latency of just over half a second.”

Ernst comments on the need for connectivity to be about more than what the truck onsite can do: “In the broadcast industry, I feel many are still stuck in the SDI way of thinking. They think when they take an SDI truck and they make it a IP truck by replaying the SDI with 10G infrastructure they’re IP-based. But the reality is as soon as you leave that truck, you’re going to hit a bottleneck in most cases. It doesn’t scale for me. The entire philosophy of this concept was that we want to make a broadcast on data lines that are really sustainable, to be available anywhere in the world, because we literally do race almost anywhere in the world. You can’t always get these 10 Gig fibres, and we don’t want to use satellite; that’s too limiting and just not really where technology should be going.”

The Formula E trackside set up

Unconventional methods

However, to make this set up work, Ernst and his team had to encourage the championship’s partners to take a leap of faith. He was convinced of what could be done using more unconventional methods, and persuaded potential partners to join them in the experiment.

Ernst explains: “We had to take certain risks, obviously. We could have just gone to an end supplier and said, “do everything for us and we step away and when something happens we’re going to make you responsible”. But I always found that all these SLAs, they’re never going to really give you the true value back if there is a really major outage, and then you’re in charge of a contract where you have no control over the risks. I’d rather build a team around me with really good people that help me manage the risks that we’re taking.”

Tata Communications was initially surprised on Ernst’s plan for connectivity over the public internet, he notes: “Tata Communications was sceptical at first, with reason. They were really like, “it’s going to be a challenge doing this scale of a broadcast over a non-dark fibre”. I asked them, “prove to me technically why it’s not feasible” and they worked for a few months and they couldn’t tell me why this actually couldn’t work.

“Now they have a product that they’re selling to other companies, because with us they built something that works. They’re using pure internet out in the field until we hit their edge network and then we go into their private cloud which gives us more protection. The connectivity cost between our connectivity setup and a regular dark fibre is around 30% to 50% of savings. We have two lines and you do that 12 times a year, it’s a significant cost saving.”

He goes on: “There was a lot of convincing that had to happen, because everybody’s scared when something goes black, for obvious reasons. It’s a hard way for a supplier to prove to you why it’s not their fault or it’s a hard way for a client to prove why it’s the supplier’s fault. Taking down this mentality and getting rid of the finger pointing mentality of just kind of opened up this playing field where Tata Communications started working with us. Gravity Media went on the journey with us as they had not done this at this level yet; it raised Gravity Media to a new level of technology with the ideas that we developed together.”

Cultural shift

One of the main challenges in this remote production was bringing crew working remotely along on the journey, Ernst says. He refers to the racking team as an example: “The remote racking, for instance; recently I went to the Westworks studio at Gravity Media to look at the final build of it, as being mainly on site I haven’t seen it since it was finished. We walked into the racking room and Steve, the CTO there, said, “here’s where the most complaining happens”. I said, “why?”, and he said, “well, because the latency of the racking is really hard, they’re not onsite and the latency is a 100 more milliseconds or so”.

“For me, okay, but that’s just a feeling; these people have been used to racking cameras that are just literally on the end of their fibre within a couple of 100 metres. When [Formula E was] in Cape Town, the cameras were 11,000 kilometres away. I think we just need to put up a ticker [at Westworks], to tell everyone how far away their cameras are so they understand the mentality of what they’re actually trying to do here, and how amazing it is. Everybody – even the industry – needs to come a little bit along on this journey.”

“What we’ve built over the last few years is a technology stack that exists now throughout the company where we can really have a unified service and a unified quality, because we control the giant screens, we control the audio on track, we control the audio and the giant screens at your home”

The end result is giving Formula E everything it desired, says Ernst: “We do the connectivity with Tata Communications and then Gravity Media does the facilities and Timeline does RF, and we manage it all together. And yes, it’s not an end to end package where we can point at someone if we go black, but at least we can manage the risk and we can fix issues individually, because we see how everything is connected, and we see how one change affects other people and the product.”

Under the new host broadcast production, the TV compound footprint on site is now around 60% of what it used to be, according to Ernst. “We have less galleries, less people, meaning less air conditioning, less cabins. We have to build everything from scratch. We literally pull this up in a parking lot everywhere we go. Freight reduction is about 40 to 50% on last season as well.”

The new production also helps Formula E achieve its goals of increased sustainability as well. “It aligns with our values and it aligns as well with the narrative of how technology-driven this championship is. I’m a true believer in doing things that really have an impact as well,” says Ernst. “If you can truly show that we’ve made the lives of a lot of people a bit more sustainable, we’re reducing how much freight we’re taking along, that counts. We’re also saving money doing it this way – a massive amount of money – which we can invest into the editorial part for instance by doing cooler things there.”

Ernst notes: “I think the concept of building all this is to be ready for what’s next. And with the framework we now have, I am very confident that if say, Apple figures out this mixed reality glasses thing to a consumer level that gets adopted and then takes off, I have zero fear that we won’t be ready for that. All our workflows now are so digital that it is very easy for us to strap something to a car or give streams to be processed in real time to facilitate that [mixed reality] format or whatever. Now, there’s maybe many million motorsports fans that do not know that we exist, which is massive, and we need to capture all those people.

Best possible framework

“I think we’ve given our company the best possible framework for any of these new ideas to come, and we can just try something out, without having to commit a tremendous amount of infrastructure to change our workflow,” continues Ernst.

“It’s proven to be really interesting. We had some very, very hard integration tests that didn’t really go the way we wanted them to go, but then we went into Mexico and produced live TV, which was quite an amazing achievement, and shows how good the architecture is that Gravity Media helped us build with Tata Communication. There’s a tremendous amount of different people that had to pull this together and believe in it. That was quite an achievement. That’s two years of hard work getting that there.

“It’s really hard to go into the detail and explain to people how ambitious the whole thing is, because everybody is focused on cloud production. If you’re in a studio and you have 50 weeks a year you got to show, then cloud production doesn’t make a lot of sense, because you’re just spinning up something that you could buy a lot cheaper. But the real win here is that we’re doing this over the internet; we’re doing this very low latency, we’re doing this with an extremely small team compared to what some other championships would be doing.”

He concludes: “What we’ve built over the last few years is a technology stack that exists now throughout the company where we can really have a unified service and a unified quality, because we control the giant screens, we control the audio on track, we control the audio and the giant screens at your home. We control the data that goes to all those screens, we control the data that now goes into the app.

Ernst’s final comments echo the mission of Formula E since its creation: to encourage the adoption of innovative, clean electric solutions that can change the world in the fight against climate change. “And like that, it helps us to make people aware where things might fail, build efficient innovations and be agile for what comes next. That’s challenging in other bigger companies with more legacy, and where people are a bit more protective of their work.”